Deploy containerized Prometheus for gnmic telemetry

Add prometheus node to the Containerlab topology, scraping the gnmic
exporter (172.16.0.70:9273) at 5s interval with relabeling to clean
device/interface/neighbor_address/vlan/mac_address/afi_safi labels.
Validated against BGP and system metrics confirmed in #44. Existing
external Prometheus instance stays untouched in parallel.

Refs #45
This commit is contained in:
2026-07-09 14:42:53 +00:00
parent b6a49bfeac
commit 1f8a3ef6d6
3 changed files with 121 additions and 2 deletions

View File

@@ -135,6 +135,7 @@ Host-facing ports:
| dc-leaf3-8 | 172.16.0.27-32 | campus-host1 | 172.16.0.105 |
| dc-access1-4 | 172.16.0.41-44 | campus-host2 | 172.16.0.106 |
| | | gnmic | 172.16.0.70 |
| | | prometheus | 172.16.0.71 |
Gateway: `172.16.0.254`.
@@ -346,12 +347,39 @@ Subscriptions (see issue #44 for the path-selection rationale):
non-numeric leaves, so the `state-to-int` event-processor maps them to `1`/`0`
- Prometheus exporter: `http://clab-arista-evpn-fabric-gnmic:9273/metrics`
A `prometheus` container node (`prom/prometheus`, `172.16.0.71:9090`) scrapes the
gnmic exporter every `5s` and is deployed inside the topology (`clab deploy`/`clab
destroy`, self-contained — see issue #45). Metric names are kept as-is; raw
OpenConfig-flattened labels are relabeled to clean, joinable names:
| Raw label | Clean label |
| -------------------------------- | ------------------ |
| `source` | `device` |
| `interface_name` | `interface` |
| `neighbor_neighbor_address` | `neighbor_address` |
| `vlan_to_vni_vlan` / `entry_vlan` | `vlan` |
| `entry_mac_address` | `mac_address` |
| `afi_safi_afi_safi_name` | `afi_safi` |
`vni` is **not** available as a native label on any gnmic-exported metric — it
only appears as the sample *value* of
`interfaces_interface_arista_vxlan_vlan_to_vnis_vlan_to_vni_state_vni`, keyed by
`(device, interface, vlan)`. Per-VNI MAC counts require a PromQL join on `vlan`
rather than a native `vni` label.
- Config: `configs/prometheus/prometheus.yml`
- This is separate from, and does not replace, the existing external Prometheus
instance — no cutover yet, both run in parallel pending validation
```bash
# Validate gnmic is subscribed and streaming from all targets
docker logs clab-arista-evpn-fabric-gnmic
# Scrape the Prometheus exporter directly
docker exec clab-arista-evpn-fabric-gnmic wget -qO- http://localhost:9273/metrics | head
# Query the in-topology Prometheus instance
curl -s 'http://172.16.0.71:9090/api/v1/query?query=system_memory_state_used' | jq
```
## 📁 Repository Structure
@@ -375,8 +403,10 @@ arista-evpn-vxlan-clab/
│ ├── campus-leaf1.cfg … campus-leaf4.cfg
│ ├── campus-border-leaf1.cfg, campus-border-leaf2.cfg
│ ├── campus-access1.cfg, campus-access2.cfg
── gnmic/
└── gnmic-config.yml
── gnmic/
└── gnmic-config.yml
│ └── prometheus/
│ └── prometheus.yml
└── hosts/
├── README.md
├── dc-server1_interfaces … dc-server4_interfaces

View File

@@ -0,0 +1,82 @@
global:
scrape_interval: 5s
evaluation_interval: 5s
scrape_configs:
- job_name: gnmic
static_configs:
- targets: ["172.16.0.70:9273"]
metric_relabel_configs:
# device: raw label is "source" (gnmic subscription target)
- source_labels: [source]
target_label: device
regex: (.+)
replacement: $1
action: replace
- action: labeldrop
regex: source
# interface: raw label is already "interface_name" (OpenConfig key on the
# "interface" path element) — normalize to "interface" for consistency
- source_labels: [interface_name]
target_label: interface
regex: (.+)
replacement: $1
action: replace
- action: labeldrop
regex: interface_name
# BGP neighbor address: raw label is double-prefixed "neighbor_neighbor_address"
# (OpenConfig path flattening — key name matches the path element name),
# see issue #44
- source_labels: [neighbor_neighbor_address]
target_label: neighbor_address
regex: (.+)
replacement: $1
action: replace
- action: labeldrop
regex: neighbor_neighbor_address
# VLAN id: normalize the two raw spellings ("vlan_to_vni_vlan" on the
# VLAN-to-VNI mapping, "entry_vlan" on FDB entries) to a single "vlan" label
- source_labels: [vlan_to_vni_vlan]
target_label: vlan
regex: (.+)
replacement: $1
action: replace
- action: labeldrop
regex: vlan_to_vni_vlan
- source_labels: [entry_vlan]
target_label: vlan
regex: (.+)
replacement: $1
action: replace
- action: labeldrop
regex: entry_vlan
# NOTE: "vni" is NOT available as a label from any gnmic-exported metric.
# The VNI only appears as the sample *value* of
# interfaces_interface_arista_vxlan_vlan_to_vnis_vlan_to_vni_state_vni,
# keyed by (device, interface, vlan). Relabeling can only rename existing
# labels, not promote a value to a label. Per-VNI MAC counts therefore
# require a PromQL join (vlan_to_vni_state_vni joined with the FDB
# entry count, grouped by vlan) rather than a native "vni" label — see
# issue #45 for the documented gap.
# MAC address on FDB entries: drop the "entry_" prefix for consistency
- source_labels: [entry_mac_address]
target_label: mac_address
regex: (.+)
replacement: $1
action: replace
- action: labeldrop
regex: entry_mac_address
# AFI/SAFI on BGP prefix counters: drop the flattened prefix
- source_labels: [afi_safi_afi_safi_name]
target_label: afi_safi
regex: (.+)
replacement: $1
action: replace
- action: labeldrop
regex: afi_safi_afi_safi_name

View File

@@ -302,6 +302,13 @@ topology:
- configs/gnmic/gnmic-config.yml:/gnmic-config.yml:ro
cmd: --config /gnmic-config.yml --log subscribe
prometheus:
kind: linux
mgmt-ipv4: 172.16.0.71
image: prom/prometheus:v2.55.1
binds:
- configs/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml:ro
links:
# =====================================================
# DATACENTER FABRIC LINKS