Add sample environment file with Grafana and IPFabric credentials

Split weathermap generation from dashboard composition
generate_weathermap.py now emits only the weathermap panel (targets + options.weathermap) and merges it into a manually-authored dashboard base (configs/grafana/dashboard-base.json) at a reserved slot panel, keeping the base's gridPos so layout stays manual. The BGP sessions table, ports/interfaces table, throughput graphs, and template variables move out of the script entirely into that base file. Verified zero regression: merged output is identical to the prior generator-only output except for JSON key ordering, and all panel queries re-validated via /api/ds/query. Refs #52
2026-07-10 10:20:29 +00:00 · 2026-07-10 10:12:07 +00:00 · 2026-07-10 09:56:07 +00:00 · 2026-07-10 09:51:48 +00:00 · 2026-07-10 09:45:40 +00:00
7 changed files with 724 additions and 74 deletions
--- a/README.md
+++ b/README.md
@@ -367,6 +367,11 @@ only appears as the sample *value* of
 `(device, interface, vlan)`. Per-VNI MAC counts require a PromQL join on `vlan`
 rather than a native `vni` label.

+`site` (`dc`/`campus`/`core`) is also not gnmic-exported. It's added as a
+Prometheus `metric_relabel_configs` rule deriving it from `device` via the
+`<area>-<role><n>` naming convention regex (see #49), rather than repeating
+`label_replace()` in every dashboard query.
+
 - Config: `configs/prometheus/prometheus.yml`
 - This is separate from, and does not replace, the existing external Prometheus
  instance — no cutover yet, both run in parallel pending validation
--- a/configs/grafana/dashboard-base.json
+++ b/configs/grafana/dashboard-base.json
@@ -0,0 +1,197 @@
+{
+  "uid": "evpn-vxlan-fabric-weathermap",
+  "title": "EVPN/VXLAN Fabric Weathermap",
+  "tags": ["evpn", "vxlan", "weathermap", "auto-generated"],
+  "timezone": "browser",
+  "schemaVersion": 39,
+  "version": 0,
+  "templating": {
+    "list": [
+      {
+        "name": "site",
+        "type": "query",
+        "datasource": { "type": "prometheus", "uid": "__DATASOURCE_UID__" },
+        "query": { "query": "label_values(interfaces_interface_state_oper_status, site)", "refId": "site" },
+        "refresh": 2,
+        "sort": 1,
+        "multi": true,
+        "includeAll": true,
+        "current": { "text": "All", "value": "$__all" }
+      },
+      {
+        "name": "device",
+        "type": "query",
+        "datasource": { "type": "prometheus", "uid": "__DATASOURCE_UID__" },
+        "query": {
+          "query": "label_values(interfaces_interface_state_oper_status{site=~\"$site\"}, device)",
+          "refId": "device"
+        },
+        "refresh": 2,
+        "sort": 1,
+        "multi": true,
+        "includeAll": true,
+        "current": { "text": "All", "value": "$__all" }
+      }
+    ]
+  },
+  "panels": [
+    {
+      "id": 1,
+      "title": "__WEATHERMAP_SLOT__",
+      "gridPos": { "h": 24, "w": 24, "x": 0, "y": 0 }
+    },
+    {
+      "id": 2,
+      "type": "table",
+      "title": "BGP Sessions",
+      "gridPos": { "h": 10, "w": 12, "x": 0, "y": 24 },
+      "datasource": { "type": "prometheus", "uid": "__DATASOURCE_UID__" },
+      "targets": [
+        {
+          "refId": "A",
+          "datasource": { "type": "prometheus", "uid": "__DATASOURCE_UID__" },
+          "instant": true,
+          "format": "table",
+          "expr": "network_instances_network_instance_protocols_protocol_bgp_neighbors_neighbor_state_session_state{device=~\"$device\"}"
+        }
+      ],
+      "transformations": [
+        {
+          "id": "organize",
+          "options": {
+            "excludeByName": {
+              "Time": true,
+              "__name__": true,
+              "instance": true,
+              "job": true,
+              "protocol_identifier": true,
+              "protocol_name": true,
+              "subscription_name": true
+            },
+            "renameByName": {
+              "device": "Device",
+              "neighbor_address": "Neighbor Address",
+              "network_instance_name": "VRF",
+              "Value": "State"
+            }
+          }
+        }
+      ],
+      "fieldConfig": {
+        "defaults": {
+          "mappings": [
+            { "type": "value", "options": { "0": { "text": "Down", "color": "red" }, "1": { "text": "Up", "color": "green" } } }
+          ],
+          "custom": { "cellOptions": { "type": "color-background" } }
+        },
+        "overrides": [
+          {
+            "matcher": { "id": "byName", "options": "State" },
+            "properties": [
+              { "id": "custom.cellOptions", "value": { "type": "color-background" } },
+              {
+                "id": "mappings",
+                "value": [
+                  { "type": "value", "options": { "0": { "text": "Down", "color": "red" }, "1": { "text": "Up", "color": "green" } } }
+                ]
+              }
+            ]
+          }
+        ]
+      }
+    },
+    {
+      "id": 3,
+      "type": "table",
+      "title": "Ports / Interfaces",
+      "gridPos": { "h": 10, "w": 12, "x": 12, "y": 24 },
+      "datasource": { "type": "prometheus", "uid": "__DATASOURCE_UID__" },
+      "targets": [
+        {
+          "refId": "A",
+          "datasource": { "type": "prometheus", "uid": "__DATASOURCE_UID__" },
+          "instant": true,
+          "format": "table",
+          "expr": "interfaces_interface_state_oper_status{device=~\"$device\"}"
+        },
+        {
+          "refId": "B",
+          "datasource": { "type": "prometheus", "uid": "__DATASOURCE_UID__" },
+          "instant": true,
+          "format": "table",
+          "expr": "rate(interfaces_interface_state_counters_in_octets{device=~\"$device\"}[5m]) * 8"
+        },
+        {
+          "refId": "C",
+          "datasource": { "type": "prometheus", "uid": "__DATASOURCE_UID__" },
+          "instant": true,
+          "format": "table",
+          "expr": "rate(interfaces_interface_state_counters_out_octets{device=~\"$device\"}[5m]) * 8"
+        }
+      ],
+      "transformations": [
+        { "id": "merge", "options": {} },
+        {
+          "id": "organize",
+          "options": {
+            "excludeByName": {
+              "Time": true,
+              "__name__": true,
+              "instance": true,
+              "job": true,
+              "subscription_name": true
+            },
+            "renameByName": {
+              "device": "Device",
+              "interface": "Interface",
+              "Value #A": "Status",
+              "Value #B": "In (bps)",
+              "Value #C": "Out (bps)"
+            }
+          }
+        }
+      ],
+      "fieldConfig": {
+        "defaults": {},
+        "overrides": [
+          {
+            "matcher": { "id": "byName", "options": "Status" },
+            "properties": [
+              { "id": "custom.cellOptions", "value": { "type": "color-background" } },
+              {
+                "id": "mappings",
+                "value": [
+                  { "type": "value", "options": { "0": { "text": "Down", "color": "red" }, "1": { "text": "Up", "color": "green" } } }
+                ]
+              }
+            ]
+          },
+          { "matcher": { "id": "byName", "options": "In (bps)" }, "properties": [{ "id": "unit", "value": "bps" }] },
+          { "matcher": { "id": "byName", "options": "Out (bps)" }, "properties": [{ "id": "unit", "value": "bps" }] }
+        ]
+      }
+    },
+    {
+      "id": 4,
+      "type": "timeseries",
+      "title": "Throughput per Site",
+      "gridPos": { "h": 10, "w": 24, "x": 0, "y": 34 },
+      "datasource": { "type": "prometheus", "uid": "__DATASOURCE_UID__" },
+      "fieldConfig": { "defaults": { "unit": "bps" }, "overrides": [] },
+      "targets": [
+        {
+          "refId": "A",
+          "datasource": { "type": "prometheus", "uid": "__DATASOURCE_UID__" },
+          "expr": "sum by (site) (rate(interfaces_interface_state_counters_in_octets{site=~\"$site\", interface!~\"Management.*\"}[5m]) * 8)",
+          "legendFormat": "{{site}} in"
+        },
+        {
+          "refId": "B",
+          "datasource": { "type": "prometheus", "uid": "__DATASOURCE_UID__" },
+          "expr": "-sum by (site) (rate(interfaces_interface_state_counters_out_octets{site=~\"$site\", interface!~\"Management.*\"}[5m]) * 8)",
+          "legendFormat": "{{site}} out"
+        }
+      ]
+    }
+  ]
+}
--- a/configs/grafana/weathermap-dashboard.json
+++ b/configs/grafana/weathermap-dashboard.json
@@ -11,10 +11,53 @@
    "timezone": "browser",
    "schemaVersion": 39,
    "version": 0,
+    "templating": {
+      "list": [
+        {
+          "name": "site",
+          "type": "query",
+          "datasource": {
+            "type": "prometheus",
+            "uid": "cfrlusac89se8a"
+          },
+          "query": {
+            "query": "label_values(interfaces_interface_state_oper_status, site)",
+            "refId": "site"
+          },
+          "refresh": 2,
+          "sort": 1,
+          "multi": true,
+          "includeAll": true,
+          "current": {
+            "text": "All",
+            "value": "$__all"
+          }
+        },
+        {
+          "name": "device",
+          "type": "query",
+          "datasource": {
+            "type": "prometheus",
+            "uid": "cfrlusac89se8a"
+          },
+          "query": {
+            "query": "label_values(interfaces_interface_state_oper_status{site=~\"$site\"}, device)",
+            "refId": "device"
+          },
+          "refresh": 2,
+          "sort": 1,
+          "multi": true,
+          "includeAll": true,
+          "current": {
+            "text": "All",
+            "value": "$__all"
+          }
+        }
+      ]
+    },
    "panels": [
      {
        "id": 1,
-        "type": "tamirsuliman-weathermap-panel",
        "title": "Fabric Weathermap",
        "gridPos": {
          "h": 24,
@@ -22,6 +65,7 @@
          "x": 0,
          "y": 0
        },
+        "type": "tamirsuliman-weathermap-panel",
        "datasource": {
          "type": "prometheus",
          "uid": "cfrlusac89se8a"
@@ -29,8 +73,8 @@
        "targets": [
          {
            "refId": "A",
-            "expr": "min by (device) (network_instances_network_instance_protocols_protocol_bgp_neighbors_neighbor_state_session_state{network_instance_name=\"default\", device=~\"campus-spine1|dc-spine1|dc-spine2|campus-spine2|campus-border-leaf1|dc-leaf1|campus-border-leaf2|dc-leaf4|dc-leaf3|dc-leaf5|dc-border-leaf1|dc-leaf6|dc-access1|dc-access2|dc-leaf8|dc-leaf7|dc-access4|dc-access3|dc-border-leaf2|dc-leaf2|campus-leaf1|campus-access1|campus-leaf2|campus-leaf3|campus-access2|campus-leaf4|core1|core2\"})",
-            "legendFormat": "BGP {{device}}",
+            "expr": "min by (device) (interfaces_interface_state_oper_status{device=~\"campus-spine1|dc-spine1|dc-spine2|campus-spine2|campus-border-leaf1|dc-leaf1|campus-border-leaf2|dc-leaf4|dc-leaf3|dc-leaf5|dc-border-leaf1|dc-leaf6|dc-access1|dc-access2|dc-leaf8|dc-leaf7|dc-access4|dc-access3|dc-border-leaf2|dc-leaf2|campus-leaf1|campus-access1|campus-leaf2|campus-leaf3|campus-access2|campus-leaf4|core1|core2\"}) * (min by (device) (network_instances_network_instance_protocols_protocol_bgp_neighbors_neighbor_state_session_state{network_instance_name=\"default\", device=~\"campus-spine1|dc-spine1|dc-spine2|campus-spine2|campus-border-leaf1|dc-leaf1|campus-border-leaf2|dc-leaf4|dc-leaf3|dc-leaf5|dc-border-leaf1|dc-leaf6|dc-access1|dc-access2|dc-leaf8|dc-leaf7|dc-access4|dc-access3|dc-border-leaf2|dc-leaf2|campus-leaf1|campus-access1|campus-leaf2|campus-leaf3|campus-access2|campus-leaf4|core1|core2\"}) or (min by (device) (interfaces_interface_state_oper_status{device=~\"campus-spine1|dc-spine1|dc-spine2|campus-spine2|campus-border-leaf1|dc-leaf1|campus-border-leaf2|dc-leaf4|dc-leaf3|dc-leaf5|dc-border-leaf1|dc-leaf6|dc-access1|dc-access2|dc-leaf8|dc-leaf7|dc-access4|dc-access3|dc-border-leaf2|dc-leaf2|campus-leaf1|campus-access1|campus-leaf2|campus-leaf3|campus-access2|campus-leaf4|core1|core2\"}) * 0 + 1))",
+            "legendFormat": "STATUS {{device}}",
            "datasource": {
              "type": "prometheus",
              "uid": "cfrlusac89se8a"
@@ -56,7 +100,7 @@
          },
          {
            "refId": "D",
-            "expr": "count by (device, vlan) (network_instances_network_instance_fdb_mac_table_entries_entry_vlan{device=~\"campus-leaf1|campus-leaf2|campus-leaf3|campus-leaf4|dc-leaf1|dc-leaf2|dc-leaf5|dc-leaf6\"})\n* on(device, vlan) group_left(vlan_to_vni_state_vni)\ninterfaces_interface_arista_vxlan_vlan_to_vnis_vlan_to_vni_state_vni{device=~\"campus-leaf1|campus-leaf2|campus-leaf3|campus-leaf4|dc-leaf1|dc-leaf2|dc-leaf5|dc-leaf6\"}",
+            "expr": "count by (device, vlan) (network_instances_network_instance_fdb_mac_table_entries_entry_vlan{device=~\"campus-leaf1|campus-leaf2|campus-leaf3|campus-leaf4|dc-leaf1|dc-leaf2|dc-leaf5|dc-leaf6\"})\n* on(device, vlan) group_left(vlan_to_vni_state_vni)\nmax by (device, vlan) (interfaces_interface_arista_vxlan_vlan_to_vnis_vlan_to_vni_state_vni{device=~\"campus-leaf1|campus-leaf2|campus-leaf3|campus-leaf4|dc-leaf1|dc-leaf2|dc-leaf5|dc-leaf6\"})",
            "legendFormat": "VNI {{device}} {{vlan}}",
            "datasource": {
              "type": "prometheus",
@@ -90,7 +134,7 @@
                  "statusDown": "#F2495C"
                },
                "nodeIcon": null,
-                "statusQuery": "BGP campus-spine1",
+                "statusQuery": "STATUS campus-spine1",
                "nodeStatusColorTarget": "border",
                "statusValueMappings": [
                  {
@@ -146,7 +190,7 @@
                  "statusDown": "#F2495C"
                },
                "nodeIcon": null,
-                "statusQuery": "BGP dc-spine1",
+                "statusQuery": "STATUS dc-spine1",
                "nodeStatusColorTarget": "border",
                "statusValueMappings": [
                  {
@@ -202,7 +246,7 @@
                  "statusDown": "#F2495C"
                },
                "nodeIcon": null,
-                "statusQuery": "BGP dc-spine2",
+                "statusQuery": "STATUS dc-spine2",
                "nodeStatusColorTarget": "border",
                "statusValueMappings": [
                  {
@@ -258,7 +302,7 @@
                  "statusDown": "#F2495C"
                },
                "nodeIcon": null,
-                "statusQuery": "BGP campus-spine2",
+                "statusQuery": "STATUS campus-spine2",
                "nodeStatusColorTarget": "border",
                "statusValueMappings": [
                  {
@@ -314,7 +358,7 @@
                  "statusDown": "#F2495C"
                },
                "nodeIcon": null,
-                "statusQuery": "BGP campus-border-leaf1",
+                "statusQuery": "STATUS campus-border-leaf1",
                "nodeStatusColorTarget": "border",
                "statusValueMappings": [
                  {
@@ -370,7 +414,7 @@
                  "statusDown": "#F2495C"
                },
                "nodeIcon": null,
-                "statusQuery": "BGP dc-leaf1",
+                "statusQuery": "STATUS dc-leaf1",
                "nodeStatusColorTarget": "border",
                "statusValueMappings": [
                  {
@@ -433,7 +477,7 @@
                  "statusDown": "#F2495C"
                },
                "nodeIcon": null,
-                "statusQuery": "BGP campus-border-leaf2",
+                "statusQuery": "STATUS campus-border-leaf2",
                "nodeStatusColorTarget": "border",
                "statusValueMappings": [
                  {
@@ -489,7 +533,7 @@
                  "statusDown": "#F2495C"
                },
                "nodeIcon": null,
-                "statusQuery": "BGP dc-leaf4",
+                "statusQuery": "STATUS dc-leaf4",
                "nodeStatusColorTarget": "border",
                "statusValueMappings": [
                  {
@@ -545,7 +589,7 @@
                  "statusDown": "#F2495C"
                },
                "nodeIcon": null,
-                "statusQuery": "BGP dc-leaf3",
+                "statusQuery": "STATUS dc-leaf3",
                "nodeStatusColorTarget": "border",
                "statusValueMappings": [
                  {
@@ -601,7 +645,7 @@
                  "statusDown": "#F2495C"
                },
                "nodeIcon": null,
-                "statusQuery": "BGP dc-leaf5",
+                "statusQuery": "STATUS dc-leaf5",
                "nodeStatusColorTarget": "border",
                "statusValueMappings": [
                  {
@@ -664,7 +708,7 @@
                  "statusDown": "#F2495C"
                },
                "nodeIcon": null,
-                "statusQuery": "BGP dc-border-leaf1",
+                "statusQuery": "STATUS dc-border-leaf1",
                "nodeStatusColorTarget": "border",
                "statusValueMappings": [
                  {
@@ -720,7 +764,7 @@
                  "statusDown": "#F2495C"
                },
                "nodeIcon": null,
-                "statusQuery": "BGP dc-leaf6",
+                "statusQuery": "STATUS dc-leaf6",
                "nodeStatusColorTarget": "border",
                "statusValueMappings": [
                  {
@@ -783,7 +827,7 @@
                  "statusDown": "#F2495C"
                },
                "nodeIcon": null,
-                "statusQuery": "BGP dc-access1",
+                "statusQuery": "STATUS dc-access1",
                "nodeStatusColorTarget": "border",
                "statusValueMappings": [
                  {
@@ -839,7 +883,7 @@
                  "statusDown": "#F2495C"
                },
                "nodeIcon": null,
-                "statusQuery": "BGP dc-access2",
+                "statusQuery": "STATUS dc-access2",
                "nodeStatusColorTarget": "border",
                "statusValueMappings": [
                  {
@@ -895,7 +939,7 @@
                  "statusDown": "#F2495C"
                },
                "nodeIcon": null,
-                "statusQuery": "BGP dc-leaf8",
+                "statusQuery": "STATUS dc-leaf8",
                "nodeStatusColorTarget": "border",
                "statusValueMappings": [
                  {
@@ -951,7 +995,7 @@
                  "statusDown": "#F2495C"
                },
                "nodeIcon": null,
-                "statusQuery": "BGP dc-leaf7",
+                "statusQuery": "STATUS dc-leaf7",
                "nodeStatusColorTarget": "border",
                "statusValueMappings": [
                  {
@@ -1007,7 +1051,7 @@
                  "statusDown": "#F2495C"
                },
                "nodeIcon": null,
-                "statusQuery": "BGP dc-access4",
+                "statusQuery": "STATUS dc-access4",
                "nodeStatusColorTarget": "border",
                "statusValueMappings": [
                  {
@@ -1063,7 +1107,7 @@
                  "statusDown": "#F2495C"
                },
                "nodeIcon": null,
-                "statusQuery": "BGP dc-access3",
+                "statusQuery": "STATUS dc-access3",
                "nodeStatusColorTarget": "border",
                "statusValueMappings": [
                  {
@@ -1119,7 +1163,7 @@
                  "statusDown": "#F2495C"
                },
                "nodeIcon": null,
-                "statusQuery": "BGP dc-border-leaf2",
+                "statusQuery": "STATUS dc-border-leaf2",
                "nodeStatusColorTarget": "border",
                "statusValueMappings": [
                  {
@@ -1175,7 +1219,7 @@
                  "statusDown": "#F2495C"
                },
                "nodeIcon": null,
-                "statusQuery": "BGP dc-leaf2",
+                "statusQuery": "STATUS dc-leaf2",
                "nodeStatusColorTarget": "border",
                "statusValueMappings": [
                  {
@@ -1238,7 +1282,7 @@
                  "statusDown": "#F2495C"
                },
                "nodeIcon": null,
-                "statusQuery": "BGP campus-leaf1",
+                "statusQuery": "STATUS campus-leaf1",
                "nodeStatusColorTarget": "border",
                "statusValueMappings": [
                  {
@@ -1301,7 +1345,7 @@
                  "statusDown": "#F2495C"
                },
                "nodeIcon": null,
-                "statusQuery": "BGP campus-access1",
+                "statusQuery": "STATUS campus-access1",
                "nodeStatusColorTarget": "border",
                "statusValueMappings": [
                  {
@@ -1357,7 +1401,7 @@
                  "statusDown": "#F2495C"
                },
                "nodeIcon": null,
-                "statusQuery": "BGP campus-leaf2",
+                "statusQuery": "STATUS campus-leaf2",
                "nodeStatusColorTarget": "border",
                "statusValueMappings": [
                  {
@@ -1420,7 +1464,7 @@
                  "statusDown": "#F2495C"
                },
                "nodeIcon": null,
-                "statusQuery": "BGP campus-leaf3",
+                "statusQuery": "STATUS campus-leaf3",
                "nodeStatusColorTarget": "border",
                "statusValueMappings": [
                  {
@@ -1483,7 +1527,7 @@
                  "statusDown": "#F2495C"
                },
                "nodeIcon": null,
-                "statusQuery": "BGP campus-access2",
+                "statusQuery": "STATUS campus-access2",
                "nodeStatusColorTarget": "border",
                "statusValueMappings": [
                  {
@@ -1539,7 +1583,7 @@
                  "statusDown": "#F2495C"
                },
                "nodeIcon": null,
-                "statusQuery": "BGP campus-leaf4",
+                "statusQuery": "STATUS campus-leaf4",
                "nodeStatusColorTarget": "border",
                "statusValueMappings": [
                  {
@@ -1602,7 +1646,7 @@
                  "statusDown": "#F2495C"
                },
                "nodeIcon": null,
-                "statusQuery": "BGP core1",
+                "statusQuery": "STATUS core1",
                "nodeStatusColorTarget": "border",
                "statusValueMappings": [
                  {
@@ -1658,7 +1702,7 @@
                  "statusDown": "#F2495C"
                },
                "nodeIcon": null,
-                "statusQuery": "BGP core2",
+                "statusQuery": "STATUS core2",
                "nodeStatusColorTarget": "border",
                "statusValueMappings": [
                  {
@@ -3910,6 +3954,287 @@
            }
          }
        }
+      },
+      {
+        "id": 2,
+        "type": "table",
+        "title": "BGP Sessions",
+        "gridPos": {
+          "h": 10,
+          "w": 12,
+          "x": 0,
+          "y": 24
+        },
+        "datasource": {
+          "type": "prometheus",
+          "uid": "cfrlusac89se8a"
+        },
+        "targets": [
+          {
+            "refId": "A",
+            "datasource": {
+              "type": "prometheus",
+              "uid": "cfrlusac89se8a"
+            },
+            "instant": true,
+            "format": "table",
+            "expr": "network_instances_network_instance_protocols_protocol_bgp_neighbors_neighbor_state_session_state{device=~\"$device\"}"
+          }
+        ],
+        "transformations": [
+          {
+            "id": "organize",
+            "options": {
+              "excludeByName": {
+                "Time": true,
+                "__name__": true,
+                "instance": true,
+                "job": true,
+                "protocol_identifier": true,
+                "protocol_name": true,
+                "subscription_name": true
+              },
+              "renameByName": {
+                "device": "Device",
+                "neighbor_address": "Neighbor Address",
+                "network_instance_name": "VRF",
+                "Value": "State"
+              }
+            }
+          }
+        ],
+        "fieldConfig": {
+          "defaults": {
+            "mappings": [
+              {
+                "type": "value",
+                "options": {
+                  "0": {
+                    "text": "Down",
+                    "color": "red"
+                  },
+                  "1": {
+                    "text": "Up",
+                    "color": "green"
+                  }
+                }
+              }
+            ],
+            "custom": {
+              "cellOptions": {
+                "type": "color-background"
+              }
+            }
+          },
+          "overrides": [
+            {
+              "matcher": {
+                "id": "byName",
+                "options": "State"
+              },
+              "properties": [
+                {
+                  "id": "custom.cellOptions",
+                  "value": {
+                    "type": "color-background"
+                  }
+                },
+                {
+                  "id": "mappings",
+                  "value": [
+                    {
+                      "type": "value",
+                      "options": {
+                        "0": {
+                          "text": "Down",
+                          "color": "red"
+                        },
+                        "1": {
+                          "text": "Up",
+                          "color": "green"
+                        }
+                      }
+                    }
+                  ]
+                }
+              ]
+            }
+          ]
+        }
+      },
+      {
+        "id": 3,
+        "type": "table",
+        "title": "Ports / Interfaces",
+        "gridPos": {
+          "h": 10,
+          "w": 12,
+          "x": 12,
+          "y": 24
+        },
+        "datasource": {
+          "type": "prometheus",
+          "uid": "cfrlusac89se8a"
+        },
+        "targets": [
+          {
+            "refId": "A",
+            "datasource": {
+              "type": "prometheus",
+              "uid": "cfrlusac89se8a"
+            },
+            "instant": true,
+            "format": "table",
+            "expr": "interfaces_interface_state_oper_status{device=~\"$device\"}"
+          },
+          {
+            "refId": "B",
+            "datasource": {
+              "type": "prometheus",
+              "uid": "cfrlusac89se8a"
+            },
+            "instant": true,
+            "format": "table",
+            "expr": "rate(interfaces_interface_state_counters_in_octets{device=~\"$device\"}[5m]) * 8"
+          },
+          {
+            "refId": "C",
+            "datasource": {
+              "type": "prometheus",
+              "uid": "cfrlusac89se8a"
+            },
+            "instant": true,
+            "format": "table",
+            "expr": "rate(interfaces_interface_state_counters_out_octets{device=~\"$device\"}[5m]) * 8"
+          }
+        ],
+        "transformations": [
+          {
+            "id": "merge",
+            "options": {}
+          },
+          {
+            "id": "organize",
+            "options": {
+              "excludeByName": {
+                "Time": true,
+                "__name__": true,
+                "instance": true,
+                "job": true,
+                "subscription_name": true
+              },
+              "renameByName": {
+                "device": "Device",
+                "interface": "Interface",
+                "Value #A": "Status",
+                "Value #B": "In (bps)",
+                "Value #C": "Out (bps)"
+              }
+            }
+          }
+        ],
+        "fieldConfig": {
+          "defaults": {},
+          "overrides": [
+            {
+              "matcher": {
+                "id": "byName",
+                "options": "Status"
+              },
+              "properties": [
+                {
+                  "id": "custom.cellOptions",
+                  "value": {
+                    "type": "color-background"
+                  }
+                },
+                {
+                  "id": "mappings",
+                  "value": [
+                    {
+                      "type": "value",
+                      "options": {
+                        "0": {
+                          "text": "Down",
+                          "color": "red"
+                        },
+                        "1": {
+                          "text": "Up",
+                          "color": "green"
+                        }
+                      }
+                    }
+                  ]
+                }
+              ]
+            },
+            {
+              "matcher": {
+                "id": "byName",
+                "options": "In (bps)"
+              },
+              "properties": [
+                {
+                  "id": "unit",
+                  "value": "bps"
+                }
+              ]
+            },
+            {
+              "matcher": {
+                "id": "byName",
+                "options": "Out (bps)"
+              },
+              "properties": [
+                {
+                  "id": "unit",
+                  "value": "bps"
+                }
+              ]
+            }
+          ]
+        }
+      },
+      {
+        "id": 4,
+        "type": "timeseries",
+        "title": "Throughput per Site",
+        "gridPos": {
+          "h": 10,
+          "w": 24,
+          "x": 0,
+          "y": 34
+        },
+        "datasource": {
+          "type": "prometheus",
+          "uid": "cfrlusac89se8a"
+        },
+        "fieldConfig": {
+          "defaults": {
+            "unit": "bps"
+          },
+          "overrides": []
+        },
+        "targets": [
+          {
+            "refId": "A",
+            "datasource": {
+              "type": "prometheus",
+              "uid": "cfrlusac89se8a"
+            },
+            "expr": "sum by (site) (rate(interfaces_interface_state_counters_in_octets{site=~\"$site\", interface!~\"Management.*\"}[5m]) * 8)",
+            "legendFormat": "{{site}} in"
+          },
+          {
+            "refId": "B",
+            "datasource": {
+              "type": "prometheus",
+              "uid": "cfrlusac89se8a"
+            },
+            "expr": "-sum by (site) (rate(interfaces_interface_state_counters_out_octets{site=~\"$site\", interface!~\"Management.*\"}[5m]) * 8)",
+            "legendFormat": "{{site}} out"
+          }
+        ]
      }
    ]
  },
--- a/configs/prometheus/prometheus.yml
+++ b/configs/prometheus/prometheus.yml
@@ -16,6 +16,16 @@ scrape_configs:
      - action: labeldrop
        regex: source

+      # site: not exported by gnmic -- derived from the hostname naming
+      # convention (see CLAUDE.md: dc-*/campus-*/core* regex used for
+      # IPFabric site separation) here instead of repeating label_replace()
+      # in every dashboard query, see issue #49
+      - source_labels: [device]
+        target_label: site
+        regex: (campus|core|dc)-?.*
+        replacement: $1
+        action: replace
+
      # interface: raw label is already "interface_name" (OpenConfig key on the
      # "interface" path element) — normalize to "interface" for consistency
      - source_labels: [interface_name]
--- a/envrc.sample
+++ b/envrc.sample
@@ -0,0 +1,4 @@
+export GRAFANA_URL=""
+export GRAFANA_TOKEN=""
+export IPFABRIC_URL=""
+export IPFABRIC_TOKEN=""
--- a/scripts/README.md
+++ b/scripts/README.md
@@ -1,10 +1,25 @@
 # scripts/generate_weathermap.py

 Generates a [weathermap-ng](https://github.com/allamiro/grafana-network-weathermap-ng)
-panel from live IPFabric topology + gnmic/Prometheus metrics, and writes a full
-Grafana dashboard-as-code JSON to `configs/grafana/weathermap-dashboard.json`
-(committed to Gitea as the source of truth, same pattern as the retired Flow
-Panel YAML). Optionally provisions it into Grafana via the HTTP API.
+**panel only** from live IPFabric topology + gnmic/Prometheus metrics, merges
+it into a manually-authored dashboard base, and writes the merged result to
+`configs/grafana/weathermap-dashboard.json` (committed to Gitea as the build
+artifact, same pattern as the retired Flow Panel YAML). Optionally provisions
+it into Grafana via the HTTP API.
+
+## File structure (see #52)
+
+| File | What it is |
+|---|---|
+| `configs/grafana/dashboard-base.json` | **Manually authored**, committed source of truth for everything except the weathermap panel: BGP sessions table, ports/interfaces table, throughput-per-site graphs, `site`/`device` template variables, panel layout. Edit this directly for anything in those panels. Contains a reserved slot panel titled `__WEATHERMAP_SLOT__` (id `1`, `gridPos` fixed) that the script splices the generated weathermap panel into. |
+| `configs/grafana/weathermap-dashboard.json` | **Generated build artifact** — base + freshly-generated weathermap panel, merged. This is what actually gets provisioned to Grafana. Don't hand-edit; re-run the script. |
+
+The script itself only ever knows about the weathermap panel (`targets` +
+`options.weathermap`) — it has no knowledge of the other three panels or the
+template variables. That split is deliberate: topology-driven content
+(nodes/links, generated from live IPFabric+Prometheus data) is separated from
+hand-tuned dashboard composition (table layout, transformations, thresholds),
+which doesn't need to regenerate on every topology change.

 See issue #48 for the panel schema research this is built against, and its
 comment thread for the live-validation history (auth quirks, plugin id, a
@@ -16,7 +31,7 @@ anchor/regex-escaping logic).
 ```bash
 export IPFABRIC_URL=https://<ipfabric-instance>
 export IPFABRIC_TOKEN=<token>
-python3 scripts/generate_weathermap.py                 # writes the JSON only
+python3 scripts/generate_weathermap.py                 # writes the merged JSON only

 export GRAFANA_URL=https://<external-grafana-instance>
 export GRAFANA_TOKEN=<token>
@@ -24,13 +39,25 @@ export GRAFANA_DATASOURCE_UID=<prometheus-datasource-uid-in-grafana>
 python3 scripts/generate_weathermap.py --provision      # also provisions via the Grafana API
 ```

-Re-run after any topology change (`evpn-lab.clab.yml`) to regenerate and
-re-provision.
+Re-run after any topology change (`evpn-lab.clab.yml`) to regenerate the
+weathermap panel and re-merge/re-provision. The script always regenerates the
+weathermap panel fresh from live IPFabric data (no diffing/incremental
+state), so a plain re-run already picks up any topology change — there's no
+separate "detect changes" step. The only manual/operational step is
+*triggering* that re-run after a change; there's no watcher or CI hook doing
+it automatically yet.
+
+If you need to change the BGP/ports/throughput panels, template variables, or
+layout, edit `configs/grafana/dashboard-base.json` directly and re-run the
+script (or run it with `--provision` to push the edit live) — no topology
+data is needed for that path, the weathermap panel just gets regenerated
+alongside it.

 ## Environment variables

 | Variable | Required | Default | Notes |
 |---|---|---|---|
+| `--base` (CLI flag, not env) | no | `configs/grafana/dashboard-base.json` | Manual dashboard base to merge the weathermap panel into. |
 | `IPFABRIC_URL` | yes | — | e.g. `https://ipfabric.example.com` |
 | `IPFABRIC_TOKEN` | yes | — | Inventory/Snapshots read access. Sent as `X-API-Token`, not `Authorization: Bearer` — IPFabric's REST API does not use bearer auth. |
 | `IPFABRIC_SNAPSHOT` | no | `$last` | |
@@ -66,8 +93,13 @@ re-provision.
   name has no matching series is logged, not silently dropped — the actual
   fix for a real mismatch belongs in gnmic interface aliasing (#43), not in
   this script.
-7. Writes the dashboard JSON, and provisions it via `POST
-   /api/dashboards/db` if `--provision` is passed.
+7. Loads `configs/grafana/dashboard-base.json`, substitutes the real
+   Prometheus datasource UID into its `__DATASOURCE_UID__` placeholders,
+   finds the panel titled `__WEATHERMAP_SLOT__` and splices in the generated
+   weathermap panel content (keeping the slot's `gridPos`/`id`, so the manual
+   layout is never repositioned).
+8. Writes the merged dashboard JSON to `configs/grafana/weathermap-dashboard.json`,
+   and provisions it via `POST /api/dashboards/db` if `--provision` is passed.

 ## Known gaps

--- a/scripts/generate_weathermap.py
+++ b/scripts/generate_weathermap.py
@@ -1,6 +1,14 @@
 #!/usr/bin/env python3
-"""Generate a weathermap-ng panel config from IPFabric topology + gnmic/Prometheus
-metrics, and optionally provision it into a Grafana dashboard.
+"""Generate a weathermap-ng PANEL (only) from IPFabric topology + gnmic/Prometheus
+metrics, merge it into a manually-authored dashboard base, and optionally
+provision the result into Grafana.
+
+Scope (see #52): this script knows only about the weathermap panel --
+targets (node status / link tx / link rx / VXLAN tooltip queries) and
+options.weathermap (nodes/links/scale/settings). It has no knowledge of the
+BGP sessions table, ports/interfaces table, or throughput panels -- those
+live in the manually-authored `configs/grafana/dashboard-base.json` and are
+not touched by this script.

 Data sources (see gitea issues #44, #47, #48):
  - IPFabric REST API: device inventory + connectivity-matrix (topology)
@@ -8,8 +16,13 @@ Data sources (see gitea issues #44, #47, #48):
    values, used both to validate the IPFabric->gnmic interface-name mapping
    and to resolve which VTEPs/VLANs actually have VXLAN data.

+Merge: the weathermap panel is generated fresh from live data on every run
+(no diffing), then spliced into the base dashboard's reserved slot (the panel
+titled "__WEATHERMAP_SLOT__"), keeping that slot's gridPos so the manually
+authored layout is never repositioned by the generator. See #52.
+
 Usage:
-    python3 scripts/generate_weathermap.py [--output PATH] [--provision]
+    python3 scripts/generate_weathermap.py [--base PATH] [--output PATH] [--provision]

 Environment:
    IPFABRIC_URL        e.g. https://ipfabric.example.com
@@ -265,7 +278,7 @@ def build_weathermap(devices, links, interface_speeds, positions, prometheus_url
            "padding": {"horizontal": 12, "vertical": 6},
            "colors": dict(NODE_COLORS),
            "nodeIcon": None,
-            "statusQuery": f"BGP {host}",
+            "statusQuery": f"STATUS {host}",
            "nodeStatusColorTarget": "border",
            "statusValueMappings": [dict(m) for m in STATUS_VALUE_MAPPINGS],
            "anchors": {a: {"numLinks": anchor_counts[host][a], "numFilledLinks": 0} for a in ANCHOR.values()},
@@ -317,11 +330,26 @@ def build_weathermap(devices, links, interface_speeds, positions, prometheus_url
    interface_regex = "|".join(sorted(interface_regex_parts))

    # -- targets (one query per metric family, per task spec) --
+    #
+    # Node status (refId A): a device is "up" only if at least one interface
+    # is up AND every BGP session is up, if it has any -- see #49. The old
+    # "BGP {{device}}" query sourced raw per-neighbor session-state series,
+    # so a device with multiple neighbors collided on one legend string and
+    # only one arbitrarily won. `min by (device)` picks the *worst* session
+    # (0 beats 1), and devices with zero BGP sessions (access switches) are
+    # not penalized: the `or` fallback substitutes a constant 1 for any
+    # device present in device-up but absent from the BGP series entirely.
+    device_up = f'min by (device) (interfaces_interface_state_oper_status{{device=~"{host_regex}"}})'
+    bgp_worst = (
+        f'min by (device) (network_instances_network_instance_protocols_protocol_bgp_neighbors_neighbor_state_session_state'
+        f'{{network_instance_name="default", device=~"{host_regex}"}})'
+    )
+    bgp_ok_or_not_applicable = f'({bgp_worst} or ({device_up} * 0 + 1))'
    targets = [
        {
            "refId": "A",
-            "expr": f'min by (device) (network_instances_network_instance_protocols_protocol_bgp_neighbors_neighbor_state_session_state{{network_instance_name="default", device=~"{host_regex}"}})',
-            "legendFormat": "BGP {{device}}",
+            "expr": f'{device_up} * {bgp_ok_or_not_applicable}',
+            "legendFormat": "STATUS {{device}}",
        },
        {
            "refId": "B",
@@ -337,12 +365,18 @@ def build_weathermap(devices, links, interface_speeds, positions, prometheus_url
    if vtep_hosts:
        vtep_regex = "|".join(promql_escape(h) for h in vtep_hosts)
        # Verbatim join from #44 "VXLAN (MAC count per VNI)", scoped to VTEP nodes.
+        # RHS is wrapped in max by (device, vlan) so the join key is always
+        # unique -- a Prometheus restart or relabel change otherwise leaves
+        # the pre-restart (frozen) and post-restart series briefly coexisting
+        # within the 5m staleness window, both matching the same (device,
+        # vlan) group, which trips PromQL's "many-to-many matching not
+        # allowed" error. See #50.
        targets.append({
            "refId": "D",
            "expr": (
                f'count by (device, vlan) (network_instances_network_instance_fdb_mac_table_entries_entry_vlan{{device=~"{vtep_regex}"}})\n'
                f'* on(device, vlan) group_left(vlan_to_vni_state_vni)\n'
-                f'interfaces_interface_arista_vxlan_vlan_to_vnis_vlan_to_vni_state_vni{{device=~"{vtep_regex}"}}'
+                f'max by (device, vlan) (interfaces_interface_arista_vxlan_vlan_to_vnis_vlan_to_vni_state_vni{{device=~"{vtep_regex}"}})'
            ),
            "legendFormat": "VNI {{device}} {{vlan}}",
        })
@@ -375,34 +409,68 @@ def build_weathermap(devices, links, interface_speeds, positions, prometheus_url


 # --------------------------------------------------------------------------
-# Grafana provisioning
+# Weathermap panel (only) -- see #52, scope narrowed from full-dashboard
 # --------------------------------------------------------------------------

-def build_dashboard(weathermap, targets, dashboard_uid, datasource_uid, plugin_id):
+WEATHERMAP_SLOT_TITLE = "__WEATHERMAP_SLOT__"
+
+
+def build_weathermap_panel(weathermap, targets, datasource_uid, plugin_id):
+    """The weathermap panel's own content: type/datasource/targets/options.
+    Deliberately has no gridPos/id/title of its own -- those come from
+    whatever slot it's merged into (see merge_weathermap_into_base)."""
    return {
-        "dashboard": {
-            "uid": dashboard_uid,
-            "title": "EVPN/VXLAN Fabric Weathermap",
-            "tags": ["evpn", "vxlan", "weathermap", "auto-generated"],
-            "timezone": "browser",
-            "schemaVersion": 39,
-            "version": 0,
-            "panels": [
-                {
-                    "id": 1,
-                    "type": plugin_id,
-                    "title": "Fabric Weathermap",
-                    "gridPos": {"h": 24, "w": 24, "x": 0, "y": 0},
-                    "datasource": {"type": "prometheus", "uid": datasource_uid},
-                    "targets": [dict(t, datasource={"type": "prometheus", "uid": datasource_uid}) for t in targets],
-                    "options": {"weathermap": weathermap},
-                }
-            ],
-        },
-        "overwrite": True,
+        "type": plugin_id,
+        "datasource": {"type": "prometheus", "uid": datasource_uid},
+        "targets": [dict(t, datasource={"type": "prometheus", "uid": datasource_uid}) for t in targets],
+        "options": {"weathermap": weathermap},
    }


+# --------------------------------------------------------------------------
+# Manual base dashboard + merge (see #52)
+# --------------------------------------------------------------------------
+
+def load_base_dashboard(path):
+    with open(path) as f:
+        return json.load(f)
+
+
+def substitute_placeholders(obj, replacements):
+    """Recursively replace exact-match string placeholders (e.g. the
+    datasource UID token) anywhere in the manually-authored base JSON."""
+    if isinstance(obj, dict):
+        return {k: substitute_placeholders(v, replacements) for k, v in obj.items()}
+    if isinstance(obj, list):
+        return [substitute_placeholders(v, replacements) for v in obj]
+    if isinstance(obj, str) and obj in replacements:
+        return replacements[obj]
+    return obj
+
+
+def merge_weathermap_into_base(base_dashboard, weathermap_panel, dashboard_uid):
+    """Splice the freshly generated weathermap panel into the base
+    dashboard's reserved slot (matched by title), keeping the slot's
+    gridPos/id -- layout stays manual, the generator never repositions it."""
+    dashboard = dict(base_dashboard)
+    dashboard["uid"] = dashboard_uid
+    panels = []
+    found = False
+    for panel in dashboard.get("panels", []):
+        if panel.get("title") == WEATHERMAP_SLOT_TITLE:
+            found = True
+            merged = dict(panel)
+            merged.update(weathermap_panel)
+            merged["title"] = "Fabric Weathermap"
+            panels.append(merged)
+        else:
+            panels.append(panel)
+    if not found:
+        sys.exit(f"Base dashboard has no panel titled {WEATHERMAP_SLOT_TITLE!r} -- nowhere to merge the weathermap panel")
+    dashboard["panels"] = panels
+    return dashboard
+
+
 def provision_to_grafana(grafana_url, api_token, dashboard_payload):
    req = urllib.request.Request(
        f"{grafana_url.rstrip('/')}/api/dashboards/db",
@@ -421,10 +489,12 @@ def provision_to_grafana(grafana_url, api_token, dashboard_payload):

 def main():
    parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    parser.add_argument("--base", default="configs/grafana/dashboard-base.json",
+                         help="manually-authored dashboard base JSON (everything but the weathermap panel)")
    parser.add_argument("--output", default="configs/grafana/weathermap-dashboard.json",
-                         help="where to write the generated dashboard-as-code JSON")
+                         help="where to write the merged dashboard-as-code JSON (build artifact)")
    parser.add_argument("--provision", action="store_true",
-                         help="also POST the dashboard to the Grafana API (requires GRAFANA_* env vars)")
+                         help="also POST the merged dashboard to the Grafana API (requires GRAFANA_* env vars)")
    args = parser.parse_args()

    ipfabric_url = env("IPFABRIC_URL", required=True)
@@ -459,13 +529,20 @@ def main():
    dashboard_uid = env("GRAFANA_DASHBOARD_UID", "evpn-vxlan-fabric-weathermap")
    datasource_uid = env("GRAFANA_DATASOURCE_UID", "PROMETHEUS_DATASOURCE_UID_PLACEHOLDER")
    plugin_id = env("GRAFANA_WEATHERMAP_PLUGIN_ID", "tamirsuliman-weathermap-panel")
-    dashboard_payload = build_dashboard(weathermap, targets, dashboard_uid, datasource_uid, plugin_id)
+    weathermap_panel = build_weathermap_panel(weathermap, targets, datasource_uid, plugin_id)
+
+    print(f"Loading manual dashboard base ({args.base})...")
+    base_dashboard = load_base_dashboard(args.base)
+    base_dashboard = substitute_placeholders(base_dashboard, {"__DATASOURCE_UID__": datasource_uid})
+    dashboard = merge_weathermap_into_base(base_dashboard, weathermap_panel, dashboard_uid)
+    dashboard_payload = {"dashboard": dashboard, "overwrite": True}

    os.makedirs(os.path.dirname(args.output) or ".", exist_ok=True)
    with open(args.output, "w") as f:
        json.dump(dashboard_payload, f, indent=2)
        f.write("\n")
-    print(f"Wrote {args.output} ({len(weathermap['nodes'])} nodes, {len(weathermap['links'])} links)")
+    print(f"Wrote {args.output} ({len(weathermap['nodes'])} nodes, {len(weathermap['links'])} links, "
+          f"{len(dashboard['panels'])} panels total)")

    if args.provision:
        grafana_url = env("GRAFANA_URL", required=True)
Author	SHA1	Message	Date
Damien Arnodo	e73b789f92	Add sample environment file with Grafana and IPFabric credentials	2026-07-10 10:20:29 +00:00
Damien Arnodo	632d146368	Split weathermap generation from dashboard composition generate_weathermap.py now emits only the weathermap panel (targets + options.weathermap) and merges it into a manually-authored dashboard base (configs/grafana/dashboard-base.json) at a reserved slot panel, keeping the base's gridPos so layout stays manual. The BGP sessions table, ports/interfaces table, throughput graphs, and template variables move out of the script entirely into that base file. Verified zero regression: merged output is identical to the prior generator-only output except for JSON key ordering, and all panel queries re-validated via /api/ds/query. Refs #52	2026-07-10 10:12:07 +00:00
Damien Arnodo	dba06c1607	Exclude Management0 from per-site throughput panel gNMI telemetry traffic on the OOB management port swamped fabric traffic in the site-aggregate sums, producing a misleading ~10x in/out asymmetry that read as a real traffic anomaly. Closes #51	2026-07-10 09:56:07 +00:00
Damien Arnodo	b726848da0	Guard VNI tooltip join against transient duplicate series Wrap the group_left RHS in max by (device, vlan) so a Prometheus restart or relabel change (old/new label-set series briefly coexisting in the staleness window) can't trip PromQL's many-to-many matching error. Closes #50	2026-07-10 09:51:48 +00:00
Damien Arnodo	b03464d0df	Extend weathermap dashboard with BGP/ports tables, per-site throughput, fix node status query Fix node.statusQuery colliding legends: multi-neighbor devices previously sourced from raw per-neighbor BGP session-state series that collide under one legend, so an arbitrary neighbor won. Replace with a combined device-up * worst-BGP-session query per device, with an explicit fallback so zero-BGP devices (access switches) read as neutral/OK. Add site label via Prometheus relabel (derived from device naming convention) instead of per-query label_replace(). Add site/device template variables, BGP sessions table, ports/interfaces table (Grafana merge transform, not PromQL join), and per-site throughput graphs. Refs #49	2026-07-10 09:45:40 +00:00