Telemetry: gnmic + Prometheus + Grafana weathermap dashboard, iperf3 traffic generator #59
40
README.md
40
README.md
@@ -384,42 +384,9 @@ curl -s 'http://172.16.0.71:9090/api/v1/query?query=system_memory_state_used' |
|
|||||||
|
|
||||||
### Weathermap panel generation (dashboard-as-code)
|
### Weathermap panel generation (dashboard-as-code)
|
||||||
|
|
||||||
`scripts/generate_weathermap.py` generates a [weathermap-ng](https://github.com/allamiro/grafana-network-weathermap-ng)
|
`scripts/generate_weathermap.py` generates a weathermap-ng Grafana panel from
|
||||||
panel from live IPFabric topology + gnmic/Prometheus metrics, and writes a full
|
live IPFabric topology + gnmic/Prometheus metrics. See
|
||||||
Grafana dashboard-as-code JSON to `configs/grafana/weathermap-dashboard.json`
|
[`scripts/README.md`](scripts/README.md) for usage and details.
|
||||||
(committed to Gitea as the source of truth, same pattern as the retired Flow
|
|
||||||
Panel YAML). See issue #48 for the schema research this is built against.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
export IPFABRIC_URL=https://<ipfabric-instance>
|
|
||||||
export IPFABRIC_TOKEN=<token>
|
|
||||||
python3 scripts/generate_weathermap.py # writes the JSON only
|
|
||||||
|
|
||||||
export GRAFANA_URL=https://<external-grafana-instance>
|
|
||||||
export GRAFANA_TOKEN=<token>
|
|
||||||
export GRAFANA_DATASOURCE_UID=<prometheus-datasource-uid-in-grafana>
|
|
||||||
python3 scripts/generate_weathermap.py --provision # also provisions via the Grafana API
|
|
||||||
```
|
|
||||||
|
|
||||||
- Re-run after any topology change (`evpn-lab.clab.yml`) to regenerate and re-provision.
|
|
||||||
- Node/link topology comes from IPFabric (`/tables/inventory/devices`,
|
|
||||||
`/tables/interfaces/connectivity-matrix`); node positions are computed as a
|
|
||||||
simple site-grouped grid (dc/core/campus bands) since IPFabric has no
|
|
||||||
layout data.
|
|
||||||
- IPFabric reports abbreviated interface names (`Et11`) while gnmic exposes
|
|
||||||
full OpenConfig names (`Ethernet11`); the script aliases known prefixes
|
|
||||||
(`Et`→`Ethernet`, `Po`→`Port-Channel`, `Lo`→`Loopback`, `Vl`→`Vlan`,
|
|
||||||
`Ma`→`Management`) and cross-checks the result against the live exporter,
|
|
||||||
logging (not silently dropping) any link whose aliased name has no matching
|
|
||||||
series — the actual fix for a real mismatch belongs in gnmic interface
|
|
||||||
aliasing (#43), not in this script.
|
|
||||||
- `.100`/`.200` subinterface rows in the connectivity-matrix (802.1Q tags used
|
|
||||||
for the gold VRF stitching on Core) are excluded — they ride the same
|
|
||||||
physical port as their parent interface and gnmic only exports physical
|
|
||||||
interface counters.
|
|
||||||
- The Grafana panel plugin id (`GRAFANA_WEATHERMAP_PLUGIN_ID`, default
|
|
||||||
`allamiro-weathermap-panel`) is a placeholder — confirm it against the
|
|
||||||
actually installed plugin before provisioning.
|
|
||||||
|
|
||||||
## 📁 Repository Structure
|
## 📁 Repository Structure
|
||||||
|
|
||||||
@@ -449,6 +416,7 @@ arista-evpn-vxlan-clab/
|
|||||||
│ └── grafana/
|
│ └── grafana/
|
||||||
│ └── weathermap-dashboard.json
|
│ └── weathermap-dashboard.json
|
||||||
├── scripts/
|
├── scripts/
|
||||||
|
│ ├── README.md
|
||||||
│ └── generate_weathermap.py
|
│ └── generate_weathermap.py
|
||||||
└── hosts/
|
└── hosts/
|
||||||
├── README.md
|
├── README.md
|
||||||
|
|||||||
79
scripts/README.md
Normal file
79
scripts/README.md
Normal file
@@ -0,0 +1,79 @@
|
|||||||
|
# scripts/generate_weathermap.py
|
||||||
|
|
||||||
|
Generates a [weathermap-ng](https://github.com/allamiro/grafana-network-weathermap-ng)
|
||||||
|
panel from live IPFabric topology + gnmic/Prometheus metrics, and writes a full
|
||||||
|
Grafana dashboard-as-code JSON to `configs/grafana/weathermap-dashboard.json`
|
||||||
|
(committed to Gitea as the source of truth, same pattern as the retired Flow
|
||||||
|
Panel YAML). Optionally provisions it into Grafana via the HTTP API.
|
||||||
|
|
||||||
|
See issue #48 for the panel schema research this is built against, and its
|
||||||
|
comment thread for the live-validation history (auth quirks, plugin id, a
|
||||||
|
PromQL escaping bug, a panel-crash fix — worth reading before touching the
|
||||||
|
anchor/regex-escaping logic).
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
```bash
|
||||||
|
export IPFABRIC_URL=https://<ipfabric-instance>
|
||||||
|
export IPFABRIC_TOKEN=<token>
|
||||||
|
python3 scripts/generate_weathermap.py # writes the JSON only
|
||||||
|
|
||||||
|
export GRAFANA_URL=https://<external-grafana-instance>
|
||||||
|
export GRAFANA_TOKEN=<token>
|
||||||
|
export GRAFANA_DATASOURCE_UID=<prometheus-datasource-uid-in-grafana>
|
||||||
|
python3 scripts/generate_weathermap.py --provision # also provisions via the Grafana API
|
||||||
|
```
|
||||||
|
|
||||||
|
Re-run after any topology change (`evpn-lab.clab.yml`) to regenerate and
|
||||||
|
re-provision.
|
||||||
|
|
||||||
|
## Environment variables
|
||||||
|
|
||||||
|
| Variable | Required | Default | Notes |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `IPFABRIC_URL` | yes | — | e.g. `https://ipfabric.example.com` |
|
||||||
|
| `IPFABRIC_TOKEN` | yes | — | Inventory/Snapshots read access. Sent as `X-API-Token`, not `Authorization: Bearer` — IPFabric's REST API does not use bearer auth. |
|
||||||
|
| `IPFABRIC_SNAPSHOT` | no | `$last` | |
|
||||||
|
| `PROMETHEUS_URL` | no | `http://172.16.0.71:9090` | The **in-topology** Prometheus (see #45), used at generation time to validate the IPFabric→gnmic interface alias and to discover live VTEP/VLAN pairs. This does not have to be the same instance Grafana queries at render time — see below. |
|
||||||
|
| `GRAFANA_URL` | with `--provision` | — | |
|
||||||
|
| `GRAFANA_TOKEN` | with `--provision` | — | Grafana Service Account token, sent as `Authorization: Bearer`. |
|
||||||
|
| `GRAFANA_DATASOURCE_UID` | with `--provision` | — | UID of the Prometheus datasource in Grafana that will actually back the panel. **This must be a datasource that can see the gnmic metrics** — an unrelated/external Prometheus instance with no gnmic data will make the panel render with no values (this happened once, see #48). |
|
||||||
|
| `GRAFANA_DASHBOARD_UID` | no | `evpn-vxlan-fabric-weathermap` | |
|
||||||
|
| `GRAFANA_WEATHERMAP_PLUGIN_ID` | no | `tamirsuliman-weathermap-panel` | The installed weathermap-ng plugin id. Override if a different fork is installed — plugin ids don't always match the upstream repo name (the schema research in #48 was done against the `allamiro` fork's source; what's actually installed here is a different fork with a stricter/different runtime schema — see the `ANCHOR` handling in the script). |
|
||||||
|
|
||||||
|
## What the script does
|
||||||
|
|
||||||
|
1. Fetches device inventory + connectivity-matrix from IPFabric.
|
||||||
|
2. Filters the connectivity-matrix to physical Ethernet-to-Ethernet links
|
||||||
|
only: drops Management-plane neighbor entries and `.100`/`.200`
|
||||||
|
subinterface rows (802.1Q tags used for the gold VRF stitching on Core —
|
||||||
|
they ride the same physical port as their parent interface and gnmic only
|
||||||
|
exports physical interface counters), and dedupes the two directions
|
||||||
|
IPFabric reports for each physical link into one.
|
||||||
|
3. Computes node positions as a simple site-grouped grid (dc/core/campus
|
||||||
|
bands) — IPFabric has no layout data.
|
||||||
|
4. Builds the panel's `targets`: one PromQL query per metric family (BGP
|
||||||
|
status, interface tx, interface rx, VXLAN MAC/VNI), each with an explicit
|
||||||
|
`legendFormat` so the resolved display name is predictable.
|
||||||
|
5. Builds `nodes[]` and `links[]` referencing those resolved legend strings,
|
||||||
|
including the plugin's `anchors` tally (per-node count of link
|
||||||
|
attachments per side) and numeric anchor enum on each link side —
|
||||||
|
omitting these crashes the panel on load in the installed plugin fork,
|
||||||
|
despite the (fork-specific) schema research saying it's safe to skip.
|
||||||
|
6. Cross-checks every IPFabric interface name, aliased to gnmic's naming
|
||||||
|
(`Et`→`Ethernet`, `Po`→`Port-Channel`, `Lo`→`Loopback`, `Vl`→`Vlan`,
|
||||||
|
`Ma`→`Management`), against the live exporter. Any link whose aliased
|
||||||
|
name has no matching series is logged, not silently dropped — the actual
|
||||||
|
fix for a real mismatch belongs in gnmic interface aliasing (#43), not in
|
||||||
|
this script.
|
||||||
|
7. Writes the dashboard JSON, and provisions it via `POST
|
||||||
|
/api/dashboards/db` if `--provision` is passed.
|
||||||
|
|
||||||
|
## Known gaps
|
||||||
|
|
||||||
|
- **VXLAN MAC-per-VNI target**: the `vlan` join key used to correlate
|
||||||
|
VLAN→VNI mapping with FDB entries (query verbatim from #44) doesn't
|
||||||
|
actually match on live data for any VTEP node — likely an Arista
|
||||||
|
internal-VLAN-vs-front-panel-VLAN translation the OpenConfig paths don't
|
||||||
|
reconcile. Only affects the decorative per-VTEP tooltip metric, not
|
||||||
|
node/link status or traffic coloring. Tracked in #44, not fixed here.
|
||||||
@@ -20,8 +20,8 @@ Environment:
|
|||||||
GRAFANA_TOKEN required with --provision
|
GRAFANA_TOKEN required with --provision
|
||||||
GRAFANA_DATASOURCE_UID Prometheus datasource UID in Grafana, required with --provision
|
GRAFANA_DATASOURCE_UID Prometheus datasource UID in Grafana, required with --provision
|
||||||
GRAFANA_DASHBOARD_UID default "evpn-vxlan-fabric-weathermap"
|
GRAFANA_DASHBOARD_UID default "evpn-vxlan-fabric-weathermap"
|
||||||
GRAFANA_WEATHERMAP_PLUGIN_ID default "allamiro-weathermap-panel" -- confirm
|
GRAFANA_WEATHERMAP_PLUGIN_ID default "tamirsuliman-weathermap-panel" -- override
|
||||||
against the actually installed plugin id before provisioning.
|
if a different weathermap-ng fork/plugin id is installed.
|
||||||
"""
|
"""
|
||||||
import argparse
|
import argparse
|
||||||
import json
|
import json
|
||||||
@@ -458,7 +458,7 @@ def main():
|
|||||||
|
|
||||||
dashboard_uid = env("GRAFANA_DASHBOARD_UID", "evpn-vxlan-fabric-weathermap")
|
dashboard_uid = env("GRAFANA_DASHBOARD_UID", "evpn-vxlan-fabric-weathermap")
|
||||||
datasource_uid = env("GRAFANA_DATASOURCE_UID", "PROMETHEUS_DATASOURCE_UID_PLACEHOLDER")
|
datasource_uid = env("GRAFANA_DATASOURCE_UID", "PROMETHEUS_DATASOURCE_UID_PLACEHOLDER")
|
||||||
plugin_id = env("GRAFANA_WEATHERMAP_PLUGIN_ID", "allamiro-weathermap-panel")
|
plugin_id = env("GRAFANA_WEATHERMAP_PLUGIN_ID", "tamirsuliman-weathermap-panel")
|
||||||
dashboard_payload = build_dashboard(weathermap, targets, dashboard_uid, datasource_uid, plugin_id)
|
dashboard_payload = build_dashboard(weathermap, targets, dashboard_uid, datasource_uid, plugin_id)
|
||||||
|
|
||||||
os.makedirs(os.path.dirname(args.output) or ".", exist_ok=True)
|
os.makedirs(os.path.dirname(args.output) or ".", exist_ok=True)
|
||||||
|
|||||||
Reference in New Issue
Block a user