Without -R, iperf3's client sends and the server only receives, so
traffic ran campus-host -> dc-server -- backwards for what these roles
represent: dc-serverN is the DC-hosted service, campus-hostN is the
consumer, and a real service delivers data to the consumer rather than
just absorbing uploads from it.
Refs #58
Z side queried z_host's rx, which measures the same A->Z flow as A
side's tx (same traffic, counted at each end) -- the Z->A direction
was never queried by either side. Use z_host's own tx instead, so
each side surfaces its own (opposite) direction.
Confirmed against the iperf3 traffic generator: DC access-leaf links
read ~500Kbps (ACK-only) before the fix despite ~18-20Mbps of real
downstream traffic to dc-server2/4, because that flow ran leaf->access
(Z->A), the unqueried direction. Campus links looked correct only by
coincidence -- campus hosts are the iperf3 clients, so their upload
traffic happens to run access->leaf (A->Z), the direction that was
queried.
Refs #57
Same content, restructured for scanning: tables instead of long
paragraphs (credentials, troubleshooting, env vars), shorter sentences,
deep rationale trimmed to a phrase + issue reference instead of inline
essays. Cut from 258 to 135 lines.
Added assets/weathermap-pipeline.svg showing the IPFabric/Prometheus ->
generator -> merge (with the manual base) -> Grafana flow, referenced at
the top of the README.
Full reset validation: deleted weathermap-dashboard.json and the live
Grafana dashboard (dashboard-base.json untouched, it's the manual part).
Regenerated from scratch and reprovisioned -- confirms the first-ever-run
path (0 live positions found, full layered layout applied, all 4 panels'
queries resolve).
Documented the whole flow in scripts/README.md as a start-to-finish
how-to: credential acquisition (UI steps for the fiddly API paths --
Grafana service account token, datasource UID, plugin id), dry run,
provision, verification (UI walkthrough + API/Explore alternative),
re-running after topology changes, and a troubleshooting table. Added
GRAFANA_DATASOURCE_UID and the optional env vars to envrc.sample.
Confirmed root cause against the installed plugin's real module.js (not
the #48 schema doc, which doesn't cover this): node height is driven by
max(anchors[Left].numLinks, anchors[Right].numLinks) whenever
compactVerticalLinks is false, which is why campus-leaf1 (3 links) and
campus-leaf2 (4 links) rendered at different heights despite the same
role. Width is unaffected by link count -- it's always recomputed from
the label text at render time, no override field exists.
Set compactVerticalLinks: true on every generated node, which makes
height a constant (fontSize + 2*padding.vertical) regardless of link
count, for every node (padding/fontSize are already uniform fabric-wide).
Refs #54
Default node position for spine/core/border-leaf/leaf/access is now a
Y-tier by role (parsed from hostname, same trust level as the site
parsing from #49) with X grouped by site within each tier, replacing
the flat site-grid heuristic.
Position specifically is now read from the live Grafana dashboard before
computing anything, so a node manually drag-and-dropped in the UI keeps
its position across regeneration -- only nodes with no live position yet
(new devices, or first-ever run) get the layered default. Devices removed
from the topology are simply absent from this run's output, so no
orphaned position/link reference is possible.
Refs #53
generate_weathermap.py now emits only the weathermap panel (targets +
options.weathermap) and merges it into a manually-authored dashboard base
(configs/grafana/dashboard-base.json) at a reserved slot panel, keeping
the base's gridPos so layout stays manual. The BGP sessions table,
ports/interfaces table, throughput graphs, and template variables move
out of the script entirely into that base file.
Verified zero regression: merged output is identical to the prior
generator-only output except for JSON key ordering, and all panel
queries re-validated via /api/ds/query.
Refs #52
gNMI telemetry traffic on the OOB management port swamped fabric traffic
in the site-aggregate sums, producing a misleading ~10x in/out asymmetry
that read as a real traffic anomaly.
Closes#51
Wrap the group_left RHS in max by (device, vlan) so a Prometheus restart
or relabel change (old/new label-set series briefly coexisting in the
staleness window) can't trip PromQL's many-to-many matching error.
Closes#50
Fix node.statusQuery colliding legends: multi-neighbor devices previously
sourced from raw per-neighbor BGP session-state series that collide under
one legend, so an arbitrary neighbor won. Replace with a combined
device-up * worst-BGP-session query per device, with an explicit fallback
so zero-BGP devices (access switches) read as neutral/OK.
Add site label via Prometheus relabel (derived from device naming
convention) instead of per-query label_replace(). Add site/device template
variables, BGP sessions table, ports/interfaces table (Grafana merge
transform, not PromQL join), and per-site throughput graphs.
Refs #49
Root README now just links to it. Also updates the default
GRAFANA_WEATHERMAP_PLUGIN_ID to the fork actually installed
(tamirsuliman-weathermap-panel), confirmed while validating #48.
Refs #48
tamirsuliman-weathermap-panel (the plugin actually installed, not the
allamiro fork the schema research was done against) requires node.anchors
-- a numLinks/numFilledLinks tally per anchor position -- and expects
link.sides.*.anchor as the plugin's numeric TS enum value (Center=0,
Top=1, Bottom=2, Left=3, Right=4), not an anchor name string. Omitting
anchors entirely (per the "safe to omit, auto-defaulted" schema-research
note, which turns out to be fork-specific) crashed the panel on load with
"TypeError: undefined is not an object (evaluating 'i.anchors[0]')".
Enum values reverse-engineered from the installed plugin's module.js.
Refs #48
re.escape() escapes '-' as '\-', which Python's regex engine accepts
but PromQL's RE2 engine rejects outright (every hostname in this lab
is hyphenated, e.g. dc-leaf1). All 4 panel targets were failing with
"unknown escape sequence U+002D" until now. Also swaps the dashboard
to the newly-created NetLab datasource, which actually has gnmic data
(the previously-wired external Prometheus datasource had none).
Refs #48
Adds scripts/generate_weathermap.py: fetches device inventory and
connectivity-matrix from IPFabric, cross-checks IPFabric's abbreviated
interface names against the live gnmic exporter, and emits a Grafana
dashboard-as-code JSON (configs/grafana/weathermap-dashboard.json) with
the weathermap-ng panel wired to explicit legendFormat-resolved queries
for BGP status, interface tx/rx, and VXLAN MAC-per-VNI.
Refs #48
Add prometheus node to the Containerlab topology, scraping the gnmic
exporter (172.16.0.70:9273) at 5s interval with relabeling to clean
device/interface/neighbor_address/vlan/mac_address/afi_safi labels.
Validated against BGP and system metrics confirmed in #44. Existing
external Prometheus instance stays untouched in parallel.
Refs #45
Add gnmic container node subscribing to interface, BGP, system, and
VXLAN gNMI paths across all Arista cEOS nodes, exposed via a
Prometheus exporter. gNMI runs in plaintext (no SSL profile on
management api gnmi), so insecure:true is used instead of TLS.
Refs #43, Refs #44
BusyBox ifup translates 'gateway X' into 'ip route add default via X'
and aborts the whole ifup run with RC=1 when that command fails with
'File exists' — which always happens on first boot because the docker
management bridge has already installed its own default via eth0. As a
result the 'post-up ip route replace default' was never executed and
the host kept the management default.
Remove the 'gateway' line so ifup only runs the idempotent 'post-up ip
route replace default via <fabric-gw>' and the fabric default wins.
Make hosts/campus-host{1,2}_interfaces the source of truth for the
campus host IP and default route, and have clab simply invoke 'ifup
eth1' at container start to apply it. Previously the bound interfaces
file was unused and the real config lived in the YAML exec block,
which was misleading.
BusyBox ifup in the network-multitool image needs 'address' plus
'netmask' rather than a CIDR, so split the address accordingly. Also
add 'post-up ip route replace default via <fabric-gw>' so the fabric
default overrides the management DHCP default even when one is already
installed.
The 'ip route add default via <fabric-gw>' exec command silently failed
on campus-host1 and campus-host2 because the management DHCP on eth0
had already installed a default via 172.16.0.254. As a result, traffic
leaving the host for other fabric subnets was sent out the management
interface instead of the EVPN fabric, breaking end-to-end ping.
Switch to 'ip route replace' so the fabric gateway overrides whatever
default is installed at container start.
With 'no bgp default ipv4-unicast' set at the router level, VRF gold
eBGP/iBGP neighbors were establishing but not exchanging any IPv4
prefixes, breaking inter-fabric transit between DC and Campus. Add an
explicit 'address-family ipv4' block with 'neighbor X activate' under
'vrf gold' on both cores and all four border leafs.
Also drop 'redistribute learned' from the border leaf VRF gold stanza:
it is not a valid command in that context and was silently stripped by
EOS.
- README: rewritten node inventory, AS map, addressing plan
(management, Lo0/Lo1, P2P, hosts), VNI/RD/RT tables, control-plane
summary and end-to-end Campus <-> DC test procedures through the
Core (VRF gold stitching).
- hosts/README: document the two new Campus host configurations.
- assets/arista-evpn-fabric.svg: new three-zone layout (Campus, Core,
DC) with legend.
- evpn-lab.clab.yml.annotations.json: reposition nodes and add zone
labels so the ContainerLab graph matches the extended topology.
core1/core2 provide L3 transit between DC and Campus fabrics. Each
physical link toward a Border Leaf is subinterfaced (.100 default,
.200 VRF gold). OSPF area 0 in default VRF, eBGP to DC BLs (65005)
and Campus BLs (66005), iBGP between core1 and core2 via Loopback0.
VRF gold uses redistribute connected and VRF-aware eBGP sessions on
.200 subinterfaces to stitch VRF gold end-to-end across fabrics.
Both border leafs share VTEP Loopback1 10.0.255.15 and peer with DC
spines in eBGP IPv4 + EVPN. Uplinks to core1/core2 use dot1q
subinterfaces (.100 default underlay, .200 VRF gold) with OSPF area 0
and eBGP to AS 65500. VRF gold extended via vxlan vrf gold vni 100001
with RD <Lo0>:1 and RT 1:100001.
