Damien/arista-evpn-vxlan-clab
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Split weathermap generation from dashboard composition

Browse Source 
generate_weathermap.py now emits only the weathermap panel (targets +
options.weathermap) and merges it into a manually-authored dashboard base
(configs/grafana/dashboard-base.json) at a reserved slot panel, keeping
the base's gridPos so layout stays manual. The BGP sessions table,
ports/interfaces table, throughput graphs, and template variables move
out of the script entirely into that base file.

Verified zero regression: merged output is identical to the prior
generator-only output except for JSON key ordering, and all panel
queries re-validated via /api/ds/query.

Refs #52
This commit is contained in:
Damien Arnodo
2026-07-10 10:12:07 +00:00
parent dba06c1607
commit 632d146368
4 changed files with 320 additions and 229 deletions
Download Patch File Download Diff File Expand all files Collapse all files

50
scripts/README.md
View File

@@ -1,10 +1,25 @@
# scripts/generate_weathermap.py
Generates a [weathermap-ng](https://github.com/allamiro/grafana-network-weathermap-ng)
panel from live IPFabric topology + gnmic/Prometheus metrics, and writes a full
Grafana dashboard-as-code JSON to `configs/grafana/weathermap-dashboard.json`
(committed to Gitea as the source of truth, same pattern as the retired Flow
Panel YAML). Optionally provisions it into Grafana via the HTTP API.
**panel only** from live IPFabric topology + gnmic/Prometheus metrics, merges
it into a manually-authored dashboard base, and writes the merged result to
`configs/grafana/weathermap-dashboard.json` (committed to Gitea as the build
artifact, same pattern as the retired Flow Panel YAML). Optionally provisions
it into Grafana via the HTTP API.
## File structure (see #52)
| File | What it is |
|---|---|
| `configs/grafana/dashboard-base.json` | **Manually authored**, committed source of truth for everything except the weathermap panel: BGP sessions table, ports/interfaces table, throughput-per-site graphs, `site`/`device` template variables, panel layout. Edit this directly for anything in those panels. Contains a reserved slot panel titled `__WEATHERMAP_SLOT__` (id `1`, `gridPos` fixed) that the script splices the generated weathermap panel into. |
| `configs/grafana/weathermap-dashboard.json` | **Generated build artifact** — base + freshly-generated weathermap panel, merged. This is what actually gets provisioned to Grafana. Don't hand-edit; re-run the script. |
The script itself only ever knows about the weathermap panel (`targets` +
`options.weathermap`) — it has no knowledge of the other three panels or the
template variables. That split is deliberate: topology-driven content
(nodes/links, generated from live IPFabric+Prometheus data) is separated from
hand-tuned dashboard composition (table layout, transformations, thresholds),
which doesn't need to regenerate on every topology change.
See issue #48 for the panel schema research this is built against, and its
comment thread for the live-validation history (auth quirks, plugin id, a
@@ -16,7 +31,7 @@ anchor/regex-escaping logic).
```bash
export IPFABRIC_URL=https://<ipfabric-instance>
export IPFABRIC_TOKEN=<token>
python3 scripts/generate_weathermap.py # writes the JSON only
python3 scripts/generate_weathermap.py # writes the merged JSON only
export GRAFANA_URL=https://<external-grafana-instance>
export GRAFANA_TOKEN=<token>
@@ -24,13 +39,25 @@ export GRAFANA_DATASOURCE_UID=<prometheus-datasource-uid-in-grafana>
python3 scripts/generate_weathermap.py --provision # also provisions via the Grafana API
```
Re-run after any topology change (`evpn-lab.clab.yml`) to regenerate and
re-provision.
Re-run after any topology change (`evpn-lab.clab.yml`) to regenerate the
weathermap panel and re-merge/re-provision. The script always regenerates the
weathermap panel fresh from live IPFabric data (no diffing/incremental
state), so a plain re-run already picks up any topology change — there's no
separate "detect changes" step. The only manual/operational step is
*triggering* that re-run after a change; there's no watcher or CI hook doing
it automatically yet.
If you need to change the BGP/ports/throughput panels, template variables, or
layout, edit `configs/grafana/dashboard-base.json` directly and re-run the
script (or run it with `--provision` to push the edit live) — no topology
data is needed for that path, the weathermap panel just gets regenerated
alongside it.
## Environment variables
| Variable | Required | Default | Notes |
|---|---|---|---|
| `--base` (CLI flag, not env) | no | `configs/grafana/dashboard-base.json` | Manual dashboard base to merge the weathermap panel into. |
| `IPFABRIC_URL` | yes | — | e.g. `https://ipfabric.example.com` |
| `IPFABRIC_TOKEN` | yes | — | Inventory/Snapshots read access. Sent as `X-API-Token`, not `Authorization: Bearer` — IPFabric's REST API does not use bearer auth. |
| `IPFABRIC_SNAPSHOT` | no | `$last` | |
@@ -66,8 +93,13 @@ re-provision.
name has no matching series is logged, not silently dropped — the actual
fix for a real mismatch belongs in gnmic interface aliasing (#43), not in
this script.
7. Writes the dashboard JSON, and provisions it via `POST
/api/dashboards/db` if `--provision` is passed.
7. Loads `configs/grafana/dashboard-base.json`, substitutes the real
Prometheus datasource UID into its `__DATASOURCE_UID__` placeholders,
finds the panel titled `__WEATHERMAP_SLOT__` and splices in the generated
weathermap panel content (keeping the slot's `gridPos`/`id`, so the manual
layout is never repositioned).
8. Writes the merged dashboard JSON to `configs/grafana/weathermap-dashboard.json`,
and provisions it via `POST /api/dashboards/db` if `--provision` is passed.
## Known gaps