Throughput-per-site panel includes Management0 telemetry traffic, not just fabric traffic #51

Closed
opened 2026-07-10 09:55:12 +00:00 by Damien · 0 comments
Owner

The "Throughput per Site" panel added in #49 sums interfaces_interface_state_counters_{in,out}_octets over all interfaces per site, with no interface filter. Management0 (the gNMI/telemetry OOB port) dominates every site's total and is heavily asymmetric (gNMI streams notifications out to the collector continuously), producing a misleading ~10x in/out imbalance that looks like a real fabric traffic anomaly but is actually management-plane noise.

Confirmed via direct query: excluding Management0 (interface!~"Management.*") brings in/out back to comparable, plausible magnitudes for all three sites.

Fix: add interface!~"Management.*" (matching the fabric-only interface filtering already used for the weathermap link queries) to both throughput target expressions in scripts/generate_weathermap.py's build_throughput_panel.

The "Throughput per Site" panel added in #49 sums `interfaces_interface_state_counters_{in,out}_octets` over **all** interfaces per site, with no interface filter. `Management0` (the gNMI/telemetry OOB port) dominates every site's total and is heavily asymmetric (gNMI streams notifications out to the collector continuously), producing a misleading ~10x in/out imbalance that looks like a real fabric traffic anomaly but is actually management-plane noise. Confirmed via direct query: excluding `Management0` (`interface!~"Management.*"`) brings in/out back to comparable, plausible magnitudes for all three sites. Fix: add `interface!~"Management.*"` (matching the fabric-only interface filtering already used for the weathermap link queries) to both throughput target expressions in `scripts/generate_weathermap.py`'s `build_throughput_panel`.
Damien added the telemetry label 2026-07-10 09:55:39 +00:00
Damien added reference feat/telemetry 2026-07-10 13:01:54 +00:00
Sign in to join this conversation.