Deploy containerized Prometheus for gnmic telemetry
Add prometheus node to the Containerlab topology, scraping the gnmic exporter (172.16.0.70:9273) at 5s interval with relabeling to clean device/interface/neighbor_address/vlan/mac_address/afi_safi labels. Validated against BGP and system metrics confirmed in #44. Existing external Prometheus instance stays untouched in parallel. Refs #45
This commit is contained in:
34
README.md
34
README.md
@@ -135,6 +135,7 @@ Host-facing ports:
|
|||||||
| dc-leaf3-8 | 172.16.0.27-32 | campus-host1 | 172.16.0.105 |
|
| dc-leaf3-8 | 172.16.0.27-32 | campus-host1 | 172.16.0.105 |
|
||||||
| dc-access1-4 | 172.16.0.41-44 | campus-host2 | 172.16.0.106 |
|
| dc-access1-4 | 172.16.0.41-44 | campus-host2 | 172.16.0.106 |
|
||||||
| | | gnmic | 172.16.0.70 |
|
| | | gnmic | 172.16.0.70 |
|
||||||
|
| | | prometheus | 172.16.0.71 |
|
||||||
|
|
||||||
Gateway: `172.16.0.254`.
|
Gateway: `172.16.0.254`.
|
||||||
|
|
||||||
@@ -346,12 +347,39 @@ Subscriptions (see issue #44 for the path-selection rationale):
|
|||||||
non-numeric leaves, so the `state-to-int` event-processor maps them to `1`/`0`
|
non-numeric leaves, so the `state-to-int` event-processor maps them to `1`/`0`
|
||||||
- Prometheus exporter: `http://clab-arista-evpn-fabric-gnmic:9273/metrics`
|
- Prometheus exporter: `http://clab-arista-evpn-fabric-gnmic:9273/metrics`
|
||||||
|
|
||||||
|
A `prometheus` container node (`prom/prometheus`, `172.16.0.71:9090`) scrapes the
|
||||||
|
gnmic exporter every `5s` and is deployed inside the topology (`clab deploy`/`clab
|
||||||
|
destroy`, self-contained — see issue #45). Metric names are kept as-is; raw
|
||||||
|
OpenConfig-flattened labels are relabeled to clean, joinable names:
|
||||||
|
|
||||||
|
| Raw label | Clean label |
|
||||||
|
| -------------------------------- | ------------------ |
|
||||||
|
| `source` | `device` |
|
||||||
|
| `interface_name` | `interface` |
|
||||||
|
| `neighbor_neighbor_address` | `neighbor_address` |
|
||||||
|
| `vlan_to_vni_vlan` / `entry_vlan` | `vlan` |
|
||||||
|
| `entry_mac_address` | `mac_address` |
|
||||||
|
| `afi_safi_afi_safi_name` | `afi_safi` |
|
||||||
|
|
||||||
|
`vni` is **not** available as a native label on any gnmic-exported metric — it
|
||||||
|
only appears as the sample *value* of
|
||||||
|
`interfaces_interface_arista_vxlan_vlan_to_vnis_vlan_to_vni_state_vni`, keyed by
|
||||||
|
`(device, interface, vlan)`. Per-VNI MAC counts require a PromQL join on `vlan`
|
||||||
|
rather than a native `vni` label.
|
||||||
|
|
||||||
|
- Config: `configs/prometheus/prometheus.yml`
|
||||||
|
- This is separate from, and does not replace, the existing external Prometheus
|
||||||
|
instance — no cutover yet, both run in parallel pending validation
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Validate gnmic is subscribed and streaming from all targets
|
# Validate gnmic is subscribed and streaming from all targets
|
||||||
docker logs clab-arista-evpn-fabric-gnmic
|
docker logs clab-arista-evpn-fabric-gnmic
|
||||||
|
|
||||||
# Scrape the Prometheus exporter directly
|
# Scrape the Prometheus exporter directly
|
||||||
docker exec clab-arista-evpn-fabric-gnmic wget -qO- http://localhost:9273/metrics | head
|
docker exec clab-arista-evpn-fabric-gnmic wget -qO- http://localhost:9273/metrics | head
|
||||||
|
|
||||||
|
# Query the in-topology Prometheus instance
|
||||||
|
curl -s 'http://172.16.0.71:9090/api/v1/query?query=system_memory_state_used' | jq
|
||||||
```
|
```
|
||||||
|
|
||||||
## 📁 Repository Structure
|
## 📁 Repository Structure
|
||||||
@@ -375,8 +403,10 @@ arista-evpn-vxlan-clab/
|
|||||||
│ ├── campus-leaf1.cfg … campus-leaf4.cfg
|
│ ├── campus-leaf1.cfg … campus-leaf4.cfg
|
||||||
│ ├── campus-border-leaf1.cfg, campus-border-leaf2.cfg
|
│ ├── campus-border-leaf1.cfg, campus-border-leaf2.cfg
|
||||||
│ ├── campus-access1.cfg, campus-access2.cfg
|
│ ├── campus-access1.cfg, campus-access2.cfg
|
||||||
│ └── gnmic/
|
│ ├── gnmic/
|
||||||
│ └── gnmic-config.yml
|
│ │ └── gnmic-config.yml
|
||||||
|
│ └── prometheus/
|
||||||
|
│ └── prometheus.yml
|
||||||
└── hosts/
|
└── hosts/
|
||||||
├── README.md
|
├── README.md
|
||||||
├── dc-server1_interfaces … dc-server4_interfaces
|
├── dc-server1_interfaces … dc-server4_interfaces
|
||||||
|
|||||||
82
configs/prometheus/prometheus.yml
Normal file
82
configs/prometheus/prometheus.yml
Normal file
@@ -0,0 +1,82 @@
|
|||||||
|
global:
|
||||||
|
scrape_interval: 5s
|
||||||
|
evaluation_interval: 5s
|
||||||
|
|
||||||
|
scrape_configs:
|
||||||
|
- job_name: gnmic
|
||||||
|
static_configs:
|
||||||
|
- targets: ["172.16.0.70:9273"]
|
||||||
|
metric_relabel_configs:
|
||||||
|
# device: raw label is "source" (gnmic subscription target)
|
||||||
|
- source_labels: [source]
|
||||||
|
target_label: device
|
||||||
|
regex: (.+)
|
||||||
|
replacement: $1
|
||||||
|
action: replace
|
||||||
|
- action: labeldrop
|
||||||
|
regex: source
|
||||||
|
|
||||||
|
# interface: raw label is already "interface_name" (OpenConfig key on the
|
||||||
|
# "interface" path element) — normalize to "interface" for consistency
|
||||||
|
- source_labels: [interface_name]
|
||||||
|
target_label: interface
|
||||||
|
regex: (.+)
|
||||||
|
replacement: $1
|
||||||
|
action: replace
|
||||||
|
- action: labeldrop
|
||||||
|
regex: interface_name
|
||||||
|
|
||||||
|
# BGP neighbor address: raw label is double-prefixed "neighbor_neighbor_address"
|
||||||
|
# (OpenConfig path flattening — key name matches the path element name),
|
||||||
|
# see issue #44
|
||||||
|
- source_labels: [neighbor_neighbor_address]
|
||||||
|
target_label: neighbor_address
|
||||||
|
regex: (.+)
|
||||||
|
replacement: $1
|
||||||
|
action: replace
|
||||||
|
- action: labeldrop
|
||||||
|
regex: neighbor_neighbor_address
|
||||||
|
|
||||||
|
# VLAN id: normalize the two raw spellings ("vlan_to_vni_vlan" on the
|
||||||
|
# VLAN-to-VNI mapping, "entry_vlan" on FDB entries) to a single "vlan" label
|
||||||
|
- source_labels: [vlan_to_vni_vlan]
|
||||||
|
target_label: vlan
|
||||||
|
regex: (.+)
|
||||||
|
replacement: $1
|
||||||
|
action: replace
|
||||||
|
- action: labeldrop
|
||||||
|
regex: vlan_to_vni_vlan
|
||||||
|
- source_labels: [entry_vlan]
|
||||||
|
target_label: vlan
|
||||||
|
regex: (.+)
|
||||||
|
replacement: $1
|
||||||
|
action: replace
|
||||||
|
- action: labeldrop
|
||||||
|
regex: entry_vlan
|
||||||
|
|
||||||
|
# NOTE: "vni" is NOT available as a label from any gnmic-exported metric.
|
||||||
|
# The VNI only appears as the sample *value* of
|
||||||
|
# interfaces_interface_arista_vxlan_vlan_to_vnis_vlan_to_vni_state_vni,
|
||||||
|
# keyed by (device, interface, vlan). Relabeling can only rename existing
|
||||||
|
# labels, not promote a value to a label. Per-VNI MAC counts therefore
|
||||||
|
# require a PromQL join (vlan_to_vni_state_vni joined with the FDB
|
||||||
|
# entry count, grouped by vlan) rather than a native "vni" label — see
|
||||||
|
# issue #45 for the documented gap.
|
||||||
|
|
||||||
|
# MAC address on FDB entries: drop the "entry_" prefix for consistency
|
||||||
|
- source_labels: [entry_mac_address]
|
||||||
|
target_label: mac_address
|
||||||
|
regex: (.+)
|
||||||
|
replacement: $1
|
||||||
|
action: replace
|
||||||
|
- action: labeldrop
|
||||||
|
regex: entry_mac_address
|
||||||
|
|
||||||
|
# AFI/SAFI on BGP prefix counters: drop the flattened prefix
|
||||||
|
- source_labels: [afi_safi_afi_safi_name]
|
||||||
|
target_label: afi_safi
|
||||||
|
regex: (.+)
|
||||||
|
replacement: $1
|
||||||
|
action: replace
|
||||||
|
- action: labeldrop
|
||||||
|
regex: afi_safi_afi_safi_name
|
||||||
@@ -302,6 +302,13 @@ topology:
|
|||||||
- configs/gnmic/gnmic-config.yml:/gnmic-config.yml:ro
|
- configs/gnmic/gnmic-config.yml:/gnmic-config.yml:ro
|
||||||
cmd: --config /gnmic-config.yml --log subscribe
|
cmd: --config /gnmic-config.yml --log subscribe
|
||||||
|
|
||||||
|
prometheus:
|
||||||
|
kind: linux
|
||||||
|
mgmt-ipv4: 172.16.0.71
|
||||||
|
image: prom/prometheus:v2.55.1
|
||||||
|
binds:
|
||||||
|
- configs/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml:ro
|
||||||
|
|
||||||
links:
|
links:
|
||||||
# =====================================================
|
# =====================================================
|
||||||
# DATACENTER FABRIC LINKS
|
# DATACENTER FABRIC LINKS
|
||||||
|
|||||||
Reference in New Issue
Block a user