Define relevant gNMI metrics for Arista EOS #44

New Issue

Damien · 2026-06-11T10:35:05Z

Damien commented

2026-06-11 10:35:05 +00:00

Identify OpenConfig YANG paths for interface IN/OUT bytes, errors, drops, and oper-status
Identify paths for BGP session state and prefix counts (underlay + overlay)
Identify VXLAN paths: MAC count per VNI, VTEP tunnel state
Add system paths: CPU and memory utilization
Cross-reference selected paths against what the weathermap-ng panel visually represents (link throughput, node status)
Document final path list as the gnmic subscription config source of truth

Packet count (weathermap-ng)

Covered by the existing interface counters path — no separate path needed:

interfaces/interface[name=<if>]/state/counters

in-unicast-pkts, out-unicast-pkts
in-multicast-pkts, out-multicast-pkts
in-broadcast-pkts, out-broadcast-pkts
in-discards, out-discards
in-errors, out-errors

Link throughput for weathermap-ng visual (thickness/color) is computed in Prometheus via rate(in-octets[5m]) / rate(out-octets[5m]), not from the raw packet counters. The pkt/discard/error counters are for a separate health panel.

BGP status — VALIDATED ✅

/network-instances/network-instance[name=default]/protocols/protocol[identifier=BGP][name=BGP]/bgp/neighbors/neighbor[neighbor-address=<ip>]/state/session-state

Plus prefix counts:
.../afi-safis/afi-safi[afi-safi-name=<AFI_SAFI>]/state/prefixes/{received,sent,installed}

Covers underlay eBGP (IPV4_UNICAST) and overlay EVPN (L2VPN_EVPN).

Confirmed live against the gnmic Prometheus exporter (curl http://172.16.0.70:9273/metrics):

Exact metric name: network_instances_network_instance_protocols_protocol_bgp_neighbors_neighbor_state_session_state
Labels: source (device), neighbor_neighbor_address (note: double-prefixed due to OpenConfig path flattening, not neighbor_address), network_instance_name, protocol_identifier, protocol_name, subscription_name (eos-bgp)
network_instance_name includes both default and gold (VRF/overlay tenant) — multi-tenant EVPN correctly represented
core1/core2 confirmed present in the BGP data (e.g. 10.0.14.1 source=core1) — core site BGP collection works end-to-end
Raw value is a string enum (ESTABLISHED, IDLE, ACTIVE, CONNECT, OPENSENT, OPENCONFIRM...), not usable directly by the Prometheus exporter. Transformed to 1 (UP/ESTABLISHED) / 0 (anything else) via a gnmic state-to-int event-processor (see #48 for the exact processor config) — same transform also applied to oper-status, so link status can use the same 0/1 pattern as BGP node status.

System metrics — VALIDATED ✅

Confirmed live: system_memory_state_used{source="<device>",subscription_name="eos-system"} streaming correctly across all sites (dc-leaf, dc-spine, core, campus).

gnmic Prometheus exporter — endpoint confirmed

Address: 172.16.0.70:9273 (not 9804 — default assumption was wrong, corrected after live test)
Accessible via plain curl http://172.16.0.70:9273/metrics

VXLAN (MAC count per VNI) — VALIDATED metrics, join is broken, moved out of weathermap

Two independent gnmic subscriptions (VTEP nodes only, eos-vxlan), joined at query time in Prometheus:

1. VLAN→VNI mapping — path /interfaces/interface[name=Vxlan1]/arista-exp-eos-vxlan:arista-vxlan/vlan-to-vnis/vlan-to-vni. VNI is the sample value, not a label:

interfaces_interface_arista_vxlan_vlan_to_vnis_vlan_to_vni_state_vni{device="dc-leaf5",interface="Vxlan1",vlan="40"} 110040

2. FDB/MAC table entries — path /network-instances/network-instance/fdb/mac-table/entries/entry. No VNI at all, only VLAN + MAC:

network_instances_network_instance_fdb_mac_table_entries_entry_state_vlan{device="dc-border-leaf2",entry_mac_address="aa:c1:ab:23:52:71",vlan="1008",network_instance_name="default"}

3. Join query (currently broken on live data):

count by (device, vlan) (network_instances_network_instance_fdb_mac_table_entries_entry_vlan)
* on(device, vlan) group_left(vlan_to_vni_state_vni)
interfaces_interface_arista_vxlan_vlan_to_vnis_vlan_to_vni_state_vni

Confirmed broken (validated during #48 provisioning, 2026-07-09): the vlan label doesn't correlate between the two metric families on any VTEP. E.g. campus-leaf1's VLAN-to-VNI mapping is keyed vlan="50", but its FDB entries show vlan="60"/"1006"/"1007"/"4090"/"4091" — zero overlap. Likely an Arista internal-VLAN-vs-front-panel-VLAN translation that the OpenConfig paths don't reconcile. Root cause not yet understood — needs investigation (possibly a third path exposing the internal↔front-panel VLAN mapping) before the join can work.

Architecture decision: VXLAN MAC/VNI is dropped from the weathermap-ng tooltip entirely (was going to be a decorative tooltipMetrics entry in #48, removed there). Instead it'll get its own dedicated Grafana table panel, separate from the weathermap — cleaner separation of concerns (weathermap = topology/link-status/traffic only), and a raw table doesn't require the join to be fixed to be useful. No separate issue opened for this yet; tracked here as a note until it's picked up.

OSPF (core1/core2)

Deferred — out of scope for this pass. Will revisit in a future iteration.

- Identify OpenConfig YANG paths for interface IN/OUT bytes, errors, drops, and oper-status - Identify paths for BGP session state and prefix counts (underlay + overlay) - Identify VXLAN paths: MAC count per VNI, VTEP tunnel state - Add system paths: CPU and memory utilization - Cross-reference selected paths against what the weathermap-ng panel visually represents (link throughput, node status) - Document final path list as the gnmic subscription config source of truth ## Packet count (weathermap-ng) Covered by the existing interface counters path — no separate path needed: `interfaces/interface[name=<if>]/state/counters` - `in-unicast-pkts`, `out-unicast-pkts` - `in-multicast-pkts`, `out-multicast-pkts` - `in-broadcast-pkts`, `out-broadcast-pkts` - `in-discards`, `out-discards` - `in-errors`, `out-errors` Link throughput for weathermap-ng visual (thickness/color) is computed in Prometheus via `rate(in-octets[5m])` / `rate(out-octets[5m])`, not from the raw packet counters. The pkt/discard/error counters are for a separate health panel. ## BGP status — VALIDATED ✅ `/network-instances/network-instance[name=default]/protocols/protocol[identifier=BGP][name=BGP]/bgp/neighbors/neighbor[neighbor-address=<ip>]/state/session-state` Plus prefix counts: `.../afi-safis/afi-safi[afi-safi-name=<AFI_SAFI>]/state/prefixes/{received,sent,installed}` Covers underlay eBGP (IPV4_UNICAST) and overlay EVPN (L2VPN_EVPN). Confirmed live against the gnmic Prometheus exporter (`curl http://172.16.0.70:9273/metrics`): - Exact metric name: `network_instances_network_instance_protocols_protocol_bgp_neighbors_neighbor_state_session_state` - Labels: `source` (device), `neighbor_neighbor_address` (note: double-prefixed due to OpenConfig path flattening, not `neighbor_address`), `network_instance_name`, `protocol_identifier`, `protocol_name`, `subscription_name` (`eos-bgp`) - `network_instance_name` includes both `default` and `gold` (VRF/overlay tenant) — multi-tenant EVPN correctly represented - **core1/core2 confirmed present** in the BGP data (e.g. `10.0.14.1` source=`core1`) — core site BGP collection works end-to-end - Raw value is a string enum (`ESTABLISHED`, `IDLE`, `ACTIVE`, `CONNECT`, `OPENSENT`, `OPENCONFIRM`...), not usable directly by the Prometheus exporter. Transformed to `1` (UP/ESTABLISHED) / `0` (anything else) via a gnmic `state-to-int` event-processor (see #48 for the exact processor config) — same transform also applied to `oper-status`, so link status can use the same 0/1 pattern as BGP node status. ## System metrics — VALIDATED ✅ Confirmed live: `system_memory_state_used{source="<device>",subscription_name="eos-system"}` streaming correctly across all sites (dc-leaf, dc-spine, core, campus). ## gnmic Prometheus exporter — endpoint confirmed - Address: `172.16.0.70:9273` (not `9804` — default assumption was wrong, corrected after live test) - Accessible via plain `curl http://172.16.0.70:9273/metrics` ## VXLAN (MAC count per VNI) — VALIDATED metrics, join is broken, moved out of weathermap Two independent gnmic subscriptions (VTEP nodes only, `eos-vxlan`), joined at query time in Prometheus: **1. VLAN→VNI mapping** — path `/interfaces/interface[name=Vxlan1]/arista-exp-eos-vxlan:arista-vxlan/vlan-to-vnis/vlan-to-vni`. VNI is the **sample value**, not a label: ``` interfaces_interface_arista_vxlan_vlan_to_vnis_vlan_to_vni_state_vni{device="dc-leaf5",interface="Vxlan1",vlan="40"} 110040 ``` **2. FDB/MAC table entries** — path `/network-instances/network-instance/fdb/mac-table/entries/entry`. No VNI at all, only VLAN + MAC: ``` network_instances_network_instance_fdb_mac_table_entries_entry_state_vlan{device="dc-border-leaf2",entry_mac_address="aa:c1:ab:23:52:71",vlan="1008",network_instance_name="default"} ``` **3. Join query (currently broken on live data):** ```promql count by (device, vlan) (network_instances_network_instance_fdb_mac_table_entries_entry_vlan) * on(device, vlan) group_left(vlan_to_vni_state_vni) interfaces_interface_arista_vxlan_vlan_to_vnis_vlan_to_vni_state_vni ``` **Confirmed broken (validated during #48 provisioning, 2026-07-09):** the `vlan` label doesn't correlate between the two metric families on any VTEP. E.g. `campus-leaf1`'s VLAN-to-VNI mapping is keyed `vlan="50"`, but its FDB entries show `vlan="60"/"1006"/"1007"/"4090"/"4091"` — zero overlap. Likely an Arista internal-VLAN-vs-front-panel-VLAN translation that the OpenConfig paths don't reconcile. Root cause not yet understood — needs investigation (possibly a third path exposing the internal↔front-panel VLAN mapping) before the join can work. **Architecture decision:** VXLAN MAC/VNI is dropped from the weathermap-ng tooltip entirely (was going to be a decorative `tooltipMetrics` entry in #48, removed there). Instead it'll get its own dedicated Grafana table panel, separate from the weathermap — cleaner separation of concerns (weathermap = topology/link-status/traffic only), and a raw table doesn't require the join to be fixed to be useful. No separate issue opened for this yet; tracked here as a note until it's picked up. ## OSPF (core1/core2) Deferred — out of scope for this pass. Will revisit in a future iteration.

Damien added the telemetry label 2026-06-11 10:35:05 +00:00

Damien commented

2026-07-09 13:31:58 +00:00

Validated all paths live against dc-leaf1 (gnmic ... capabilities + gnmic ... get --path <path>) and wired them into configs/gnmic/gnmic-config.yml. Findings:

Interfaces — already covered by #43: /interfaces/interface/state/{oper-status,counters}.

BGP — as specified:

.../bgp/neighbors/neighbor/state/session-state
.../bgp/neighbors/neighbor/afi-safis/afi-safi/state/prefixes — confirmed both IPV4_UNICAST and L2VPN_EVPN afi-safi entries exist, each with received/sent/installed/received-pre-policy (+ Arista-augmented best-paths/best-ecmp-paths).

VXLAN — no single OpenConfig path gives "MAC count per VNI" or a dedicated "VTEP tunnel state" leaf. Used instead:

/interfaces/interface[name=Vxlan1]/arista-exp-eos-vxlan:arista-vxlan/vlan-to-vnis/vlan-to-vni (native Arista augment) for the VLAN↔VNI mapping.
/network-instances/network-instance/fdb/mac-table/entries/entry for MAC entries, tagged by VLAN.
MAC count per VNI is a Prometheus-side join (VLAN→VNI mapping × count of MAC entries per VLAN), not a raw device counter.
"VTEP tunnel state" has no dedicated leaf either — inferred from remote-learned mac-table entries (interface = remote VTEP) rather than a standalone list. Only applied to VTEP-capable nodes (leafs + border-leafs in both fabrics), not spines/access/core.

System — /system/cpus/cpu/state (per-core), /system/memory/state. Both plain OpenConfig, no surprises.

Gotcha worth flagging: oper-status and session-state are string enums (UP/DOWN, ESTABLISHED/IDLE/...). gnmic's Prometheus output silently drops non-numeric leaf values — they were missing from /metrics entirely until I added a state-to-int event-processor mapping them to 1/0. This also fixes a latent gap in the #43 config (oper-status was never actually exported).

Full metric list validated end-to-end via containerlab deploy --node-filter dc-leaf1,gnmic + scraping :9273/metrics, then torn down cleanly. OSPF still deferred per the issue description.

Validated all paths live against `dc-leaf1` (`gnmic ... capabilities` + `gnmic ... get --path <path>`) and wired them into `configs/gnmic/gnmic-config.yml`. Findings: **Interfaces** — already covered by #43: `/interfaces/interface/state/{oper-status,counters}`. **BGP** — as specified: - `.../bgp/neighbors/neighbor/state/session-state` - `.../bgp/neighbors/neighbor/afi-safis/afi-safi/state/prefixes` — confirmed both `IPV4_UNICAST` and `L2VPN_EVPN` afi-safi entries exist, each with `received`/`sent`/`installed`/`received-pre-policy` (+ Arista-augmented `best-paths`/`best-ecmp-paths`). **VXLAN** — no single OpenConfig path gives "MAC count per VNI" or a dedicated "VTEP tunnel state" leaf. Used instead: - `/interfaces/interface[name=Vxlan1]/arista-exp-eos-vxlan:arista-vxlan/vlan-to-vnis/vlan-to-vni` (native Arista augment) for the VLAN↔VNI mapping. - `/network-instances/network-instance/fdb/mac-table/entries/entry` for MAC entries, tagged by VLAN. - MAC count per VNI is a Prometheus-side join (VLAN→VNI mapping × count of MAC entries per VLAN), not a raw device counter. - "VTEP tunnel state" has no dedicated leaf either — inferred from remote-learned mac-table entries (interface = remote VTEP) rather than a standalone list. Only applied to VTEP-capable nodes (leafs + border-leafs in both fabrics), not spines/access/core. **System** — `/system/cpus/cpu/state` (per-core), `/system/memory/state`. Both plain OpenConfig, no surprises. **Gotcha worth flagging**: `oper-status` and `session-state` are string enums (`UP`/`DOWN`, `ESTABLISHED`/`IDLE`/...). gnmic's Prometheus output silently drops non-numeric leaf values — they were missing from `/metrics` entirely until I added a `state-to-int` event-processor mapping them to `1`/`0`. This also fixes a latent gap in the #43 config (oper-status was never actually exported). Full metric list validated end-to-end via `containerlab deploy --node-filter dc-leaf1,gnmic` + scraping `:9273/metrics`, then torn down cleanly. OSPF still deferred per the issue description.

Damien referenced this issue from a commit

2026-07-09 13:33:44 +00:00

Integrate gnmic for gNMI telemetry collection

Damien referenced this issue

2026-07-09 14:28:18 +00:00

Configure Prometheus scrape for gnmic #45

Damien referenced this issue

2026-07-09 15:05:42 +00:00

Generate weathermap-ng nodes/links config from IPFabric topology #48

Damien referenced this issue

2026-07-09 15:27:41 +00:00

Generate weathermap-ng nodes/links config from IPFabric topology #48