Z side queried z_host's rx, which measures the same A->Z flow as A
side's tx (same traffic, counted at each end) -- the Z->A direction
was never queried by either side. Use z_host's own tx instead, so
each side surfaces its own (opposite) direction.
Confirmed against the iperf3 traffic generator: DC access-leaf links
read ~500Kbps (ACK-only) before the fix despite ~18-20Mbps of real
downstream traffic to dc-server2/4, because that flow ran leaf->access
(Z->A), the unqueried direction. Campus links looked correct only by
coincidence -- campus hosts are the iperf3 clients, so their upload
traffic happens to run access->leaf (A->Z), the direction that was
queried.
Refs #57
Full reset validation: deleted weathermap-dashboard.json and the live
Grafana dashboard (dashboard-base.json untouched, it's the manual part).
Regenerated from scratch and reprovisioned -- confirms the first-ever-run
path (0 live positions found, full layered layout applied, all 4 panels'
queries resolve).
Documented the whole flow in scripts/README.md as a start-to-finish
how-to: credential acquisition (UI steps for the fiddly API paths --
Grafana service account token, datasource UID, plugin id), dry run,
provision, verification (UI walkthrough + API/Explore alternative),
re-running after topology changes, and a troubleshooting table. Added
GRAFANA_DATASOURCE_UID and the optional env vars to envrc.sample.
Confirmed root cause against the installed plugin's real module.js (not
the #48 schema doc, which doesn't cover this): node height is driven by
max(anchors[Left].numLinks, anchors[Right].numLinks) whenever
compactVerticalLinks is false, which is why campus-leaf1 (3 links) and
campus-leaf2 (4 links) rendered at different heights despite the same
role. Width is unaffected by link count -- it's always recomputed from
the label text at render time, no override field exists.
Set compactVerticalLinks: true on every generated node, which makes
height a constant (fontSize + 2*padding.vertical) regardless of link
count, for every node (padding/fontSize are already uniform fabric-wide).
Refs #54
Default node position for spine/core/border-leaf/leaf/access is now a
Y-tier by role (parsed from hostname, same trust level as the site
parsing from #49) with X grouped by site within each tier, replacing
the flat site-grid heuristic.
Position specifically is now read from the live Grafana dashboard before
computing anything, so a node manually drag-and-dropped in the UI keeps
its position across regeneration -- only nodes with no live position yet
(new devices, or first-ever run) get the layered default. Devices removed
from the topology are simply absent from this run's output, so no
orphaned position/link reference is possible.
Refs #53
generate_weathermap.py now emits only the weathermap panel (targets +
options.weathermap) and merges it into a manually-authored dashboard base
(configs/grafana/dashboard-base.json) at a reserved slot panel, keeping
the base's gridPos so layout stays manual. The BGP sessions table,
ports/interfaces table, throughput graphs, and template variables move
out of the script entirely into that base file.
Verified zero regression: merged output is identical to the prior
generator-only output except for JSON key ordering, and all panel
queries re-validated via /api/ds/query.
Refs #52
gNMI telemetry traffic on the OOB management port swamped fabric traffic
in the site-aggregate sums, producing a misleading ~10x in/out asymmetry
that read as a real traffic anomaly.
Closes#51
Wrap the group_left RHS in max by (device, vlan) so a Prometheus restart
or relabel change (old/new label-set series briefly coexisting in the
staleness window) can't trip PromQL's many-to-many matching error.
Closes#50
Fix node.statusQuery colliding legends: multi-neighbor devices previously
sourced from raw per-neighbor BGP session-state series that collide under
one legend, so an arbitrary neighbor won. Replace with a combined
device-up * worst-BGP-session query per device, with an explicit fallback
so zero-BGP devices (access switches) read as neutral/OK.
Add site label via Prometheus relabel (derived from device naming
convention) instead of per-query label_replace(). Add site/device template
variables, BGP sessions table, ports/interfaces table (Grafana merge
transform, not PromQL join), and per-site throughput graphs.
Refs #49
tamirsuliman-weathermap-panel (the plugin actually installed, not the
allamiro fork the schema research was done against) requires node.anchors
-- a numLinks/numFilledLinks tally per anchor position -- and expects
link.sides.*.anchor as the plugin's numeric TS enum value (Center=0,
Top=1, Bottom=2, Left=3, Right=4), not an anchor name string. Omitting
anchors entirely (per the "safe to omit, auto-defaulted" schema-research
note, which turns out to be fork-specific) crashed the panel on load with
"TypeError: undefined is not an object (evaluating 'i.anchors[0]')".
Enum values reverse-engineered from the installed plugin's module.js.
Refs #48
re.escape() escapes '-' as '\-', which Python's regex engine accepts
but PromQL's RE2 engine rejects outright (every hostname in this lab
is hyphenated, e.g. dc-leaf1). All 4 panel targets were failing with
"unknown escape sequence U+002D" until now. Also swaps the dashboard
to the newly-created NetLab datasource, which actually has gnmic data
(the previously-wired external Prometheus datasource had none).
Refs #48
Adds scripts/generate_weathermap.py: fetches device inventory and
connectivity-matrix from IPFabric, cross-checks IPFabric's abbreviated
interface names against the live gnmic exporter, and emits a Grafana
dashboard-as-code JSON (configs/grafana/weathermap-dashboard.json) with
the weathermap-ng panel wired to explicit legendFormat-resolved queries
for BGP status, interface tx/rx, and VXLAN MAC-per-VNI.
Refs #48
Add prometheus node to the Containerlab topology, scraping the gnmic
exporter (172.16.0.70:9273) at 5s interval with relabeling to clean
device/interface/neighbor_address/vlan/mac_address/afi_safi labels.
Validated against BGP and system metrics confirmed in #44. Existing
external Prometheus instance stays untouched in parallel.
Refs #45
Add gnmic container node subscribing to interface, BGP, system, and
VXLAN gNMI paths across all Arista cEOS nodes, exposed via a
Prometheus exporter. gNMI runs in plaintext (no SSL profile on
management api gnmi), so insecure:true is used instead of TLS.
Refs #43, Refs #44
With 'no bgp default ipv4-unicast' set at the router level, VRF gold
eBGP/iBGP neighbors were establishing but not exchanging any IPv4
prefixes, breaking inter-fabric transit between DC and Campus. Add an
explicit 'address-family ipv4' block with 'neighbor X activate' under
'vrf gold' on both cores and all four border leafs.
Also drop 'redistribute learned' from the border leaf VRF gold stanza:
it is not a valid command in that context and was silently stripped by
EOS.
core1/core2 provide L3 transit between DC and Campus fabrics. Each
physical link toward a Border Leaf is subinterfaced (.100 default,
.200 VRF gold). OSPF area 0 in default VRF, eBGP to DC BLs (65005)
and Campus BLs (66005), iBGP between core1 and core2 via Loopback0.
VRF gold uses redistribute connected and VRF-aware eBGP sessions on
.200 subinterfaces to stitch VRF gold end-to-end across fabrics.
Both border leafs share VTEP Loopback1 10.0.255.15 and peer with DC
spines in eBGP IPv4 + EVPN. Uplinks to core1/core2 use dot1q
subinterfaces (.100 default underlay, .200 VRF gold) with OSPF area 0
and eBGP to AS 65500. VRF gold extended via vxlan vrf gold vni 100001
with RD <Lo0>:1 and RT 1:100001.
Update configuration files for all leaf and spine switches to include a
new static route for the 100.64.0.0/10 prefix.
The route points to the next-hop gateway 172.16.0.254, ensuring proper
reachability for the Tailnet range across the network fabric.
This commit updates the management interface configuration across all leaf and spine switches to align with the target environment requirements.
Changes include:
- Changing the primary management interface from `Management1` to `Management0`.
- Removing the `vrf mgmt` assignment, moving the interface to the default global routing table.
- Explicitly configuring `lldp management-address Management0` to ensure the correct management IP is advertised to neighbors.
Enables the gNMI (gRPC Network Management Interface) API across all leaf
and spine switches to allow for telemetry streaming and programmatic
device management.
Configuration details:
- Transport: grpc default
- Provider: eos-native
This enables BGP and IP forwarding on spine switches. Without this command,
BGP sessions cannot be established and routing is disabled. This is essential
for the underlay fabric to function properly.
This enables BGP and IP forwarding on spine switches. Without this command,
BGP sessions cannot be established and routing is disabled. This is essential
for the underlay fabric to function properly.
