Compare commits
31 Commits
baa95a33c0
...
31014bdfdf
| Author | SHA1 | Date | |
|---|---|---|---|
| 31014bdfdf | |||
| 5e5d6b45df | |||
| 4719a77f21 | |||
| 08124c8a1c | |||
| 8349791630 | |||
| 3c3b28c987 | |||
| 5b12894289 | |||
|
|
0f3aa27566 | ||
| 1cbea38522 | |||
| 64a8287518 | |||
| 045179e098 | |||
| 998014d144 | |||
| 8fb83b9446 | |||
| 9b53ab0683 | |||
| fcd7ea2567 | |||
| e73b789f92 | |||
| 632d146368 | |||
| dba06c1607 | |||
| b726848da0 | |||
| b03464d0df | |||
| 3c680a2b17 | |||
| 36c93d9ecd | |||
| 0aff25f509 | |||
| f711210379 | |||
| 1f8a3ef6d6 | |||
| b6a49bfeac | |||
| 71cc8de7d5 | |||
|
|
d12e057bc8 | ||
|
|
d3b59a02e9 | ||
|
|
553c713d17 | ||
|
|
3104d7ea63 |
78
CLAUDE.md
Normal file
78
CLAUDE.md
Normal file
@@ -0,0 +1,78 @@
|
||||
# CLAUDE.md
|
||||
|
||||
## Project context
|
||||
|
||||
Arista **EVPN/VXLAN** network lab built on Containerlab + cEOS, simulating three
|
||||
interconnected zones:
|
||||
|
||||
- **DC** (Data Center): 2 spines, 8 leafs (4 MLAG/VTEP pairs), 2 MLAG border leafs, 4 access switches, 4 hosts
|
||||
- **Core**: 2 L3 transit routers (iBGP AS 65500, OSPF underlay)
|
||||
- **Campus**: 2 spines, 4 leafs (2 MLAG/VTEP pairs), 2 MLAG border leafs, 2 access switches, 2 hosts
|
||||
|
||||
A `gold` VRF is stretched end-to-end between DC and Campus through the Core (EVPN
|
||||
Type-5, stitched over eBGP IPv4 on the Core).
|
||||
|
||||
Source of truth repo: `https://gitea.arnodo.fr/Damien/arista-evpn-vxlan-clab`
|
||||
|
||||
## Naming convention
|
||||
|
||||
- Hostname format: `<area>-<role><n>` where `area` ∈ `{dc, campus, core}`
|
||||
- Examples: `dc-leaf1`, `campus-border-leaf2`, `core1`
|
||||
- This convention is used as-is for IPFabric site separation
|
||||
(regex: `^(dc|campus|core)-?.*$`)
|
||||
|
||||
## Repository structure
|
||||
|
||||
- `README.md` — overview, IP addressing, VNI mapping, control plane, quickstart
|
||||
- `TROUBLESHOOTING.md` — diagnostic procedures
|
||||
- `END_TO_END_TESTING.md` — end-to-end test scenarios
|
||||
- `evpn-lab.clab.yml` — Containerlab topology definition
|
||||
- `configs/` — Arista EOS configs per node (`<hostname>.cfg`)
|
||||
- `hosts/` — network configuration for simulated Linux hosts
|
||||
- `assets/` — diagrams (drawio/svg)
|
||||
|
||||
## Tech stack
|
||||
|
||||
- **Containerlab** for topology orchestration
|
||||
- **cEOS** (containerized Arista EOS) as the network OS
|
||||
- **IPFabric** for network assurance and site separation
|
||||
- **Gitea** (self-hosted, `gitea.arnodo.fr`) as the source of truth
|
||||
|
||||
## GitOps workflow — mandatory rule
|
||||
|
||||
**Every action on this project must be tracked by a Gitea issue.**
|
||||
|
||||
- Before starting any work (config change, topology addition, documentation, etc.),
|
||||
confirm an **issue exists** and reference it in commits (`Refs #<n>` or `Closes #<n>`).
|
||||
- If no issue covers the identified need: **create the issue first**, describe the
|
||||
problem/requirement, and only then act on it.
|
||||
- No TODO should ever be left "floating" in code, commits, or discussions: it's
|
||||
either already an issue, or it must become one immediately.
|
||||
- This `CLAUDE.md` file stays **general**: it must never contain one-off tasks.
|
||||
Action details live exclusively in issues.
|
||||
|
||||
## Style guidelines
|
||||
|
||||
- Be concise and direct — no filler, no unnecessary preamble
|
||||
- Issues: short title, straight to the point description (context, expected outcome, no fluff)
|
||||
- Commit messages: short, factual, imperative mood
|
||||
- Documentation (README, TROUBLESHOOTING, etc.): technical and dense — tables over
|
||||
prose, no marketing tone, no redundant explanations
|
||||
- Avoid restating what's already documented elsewhere — link/reference instead of duplicating
|
||||
|
||||
## Contribution conventions
|
||||
|
||||
- Atomic commits, clear message, reference to the associated issue
|
||||
- Any Arista config change must stay consistent with the IP addressing / VNI / RD
|
||||
tables documented in `README.md`
|
||||
- Any topology change (`evpn-lab.clab.yml`) must be reflected in `assets/` (diagram)
|
||||
and in the `README.md` tables
|
||||
- New validation procedures go in `END_TO_END_TESTING.md`; known issues go in
|
||||
`TROUBLESHOOTING.md`
|
||||
|
||||
## Default behavior for Claude
|
||||
|
||||
1. Before any action, identify the corresponding Gitea issue (or create one first)
|
||||
2. Stay within the scope of the issue being worked on — don't drift to other topics
|
||||
3. Strictly follow the `<area>-<role><n>` naming convention
|
||||
4. Never introduce hardcoded TODO/FIXME in repo files — always go through an issue
|
||||
329
README.md
329
README.md
@@ -4,11 +4,11 @@ An extended Arista BGP EVPN-VXLAN multi-fabric lab using ContainerLab and cEOS.
|
||||
|
||||
## 🎯 Overview
|
||||
|
||||
| Zone | Devices |
|
||||
| ------ | --------------------------------------------------------------------------------------- |
|
||||
| DC | 2 spines, 8 leafs (4 MLAG VTEPs), 2 border leafs (MLAG), 4 access switches, 4 hosts |
|
||||
| Core | 2 core routers (iBGP AS 65500, OSPF underlay with BLs, eBGP to DC & Campus BLs) |
|
||||
| Campus | 2 spines, 4 leafs (2 MLAG VTEPs), 2 border leafs (MLAG), 2 access switches, 2 hosts |
|
||||
| Zone | Devices |
|
||||
| ------ | ----------------------------------------------------------------------------------- |
|
||||
| DC | 2 spines, 8 leafs (4 MLAG VTEPs), 2 border leafs (MLAG), 4 access switches, 4 hosts |
|
||||
| Core | 2 core routers (iBGP AS 65500, OSPF underlay with BLs, eBGP to DC & Campus BLs) |
|
||||
| Campus | 2 spines, 4 leafs (2 MLAG VTEPs), 2 border leafs (MLAG), 2 access switches, 2 hosts |
|
||||
|
||||
Key design choices:
|
||||
|
||||
@@ -16,12 +16,12 @@ Key design choices:
|
||||
- **OSPF area 0 + eBGP multi-hop** between each Border Leaf pair and both Core routers (over dot1q subinterfaces: `.100` = default VRF underlay, `.200` = VRF `gold`).
|
||||
- **MLAG** everywhere there is dual-homing at the fabric layers (leaf pairs, border-leaf pairs, access → leafs, and DC host → access).
|
||||
- **Host attachment pattern**:
|
||||
- **DC hosts** (servers) are **dual-homed via LACP** to an access switch — typical DC
|
||||
server redundancy.
|
||||
- **Campus hosts** (user endpoints: PC, phone, printer) are **single-attached** to a
|
||||
Campus access switch via one plain Ethernet link. Redundancy lives at the access-switch
|
||||
layer (the access switch itself is dual-homed via LACP to its leaf MLAG pair), not at
|
||||
the host.
|
||||
- **DC hosts** (servers) are **dual-homed via LACP** to an access switch — typical DC
|
||||
server redundancy.
|
||||
- **Campus hosts** (user endpoints: PC, phone, printer) are **single-attached** to a
|
||||
Campus access switch via one plain Ethernet link. Redundancy lives at the access-switch
|
||||
layer (the access switch itself is dual-homed via LACP to its leaf MLAG pair), not at
|
||||
the host.
|
||||
- **VRF `gold`** is stretched end-to-end: DC leafs (VLAN 34 / 78) ↔ DC-BL ↔ Core ↔ Campus-BL ↔ Campus leafs (VLAN 60 / 70), all sharing L3 VNI `100001`.
|
||||
- **VLAN 50** remains defined as a campus-local L2 VXLAN stretched between the two Campus VTEPs (infrastructure-only, not wired to any host in the current topology).
|
||||
- **Convention**: L2 VNI = `110000 + vlan_id`, L3 VNI = `100001` for VRF `gold`, RT `1:100001` in both fabrics.
|
||||
@@ -64,50 +64,50 @@ docker exec -it clab-arista-evpn-fabric-dc-border-leaf1 Cli
|
||||
|
||||
### Node Inventory
|
||||
|
||||
| Zone | Role | Nodes | AS |
|
||||
| ------ | ----------------------- | ------------------------------------------------------ | ------ |
|
||||
| DC | Spine | `dc-spine1`, `dc-spine2` | 65000 |
|
||||
| DC | Leaf VTEP1 (MLAG) | `dc-leaf1`, `dc-leaf2` | 65001 |
|
||||
| DC | Leaf VTEP2 (MLAG) | `dc-leaf3`, `dc-leaf4` | 65002 |
|
||||
| DC | Leaf VTEP3 (MLAG) | `dc-leaf5`, `dc-leaf6` | 65003 |
|
||||
| DC | Leaf VTEP4 (MLAG) | `dc-leaf7`, `dc-leaf8` | 65004 |
|
||||
| DC | Border Leaf (MLAG) | `dc-border-leaf1`, `dc-border-leaf2` | 65005 |
|
||||
| DC | Access (L2-only) | `dc-access1`-`dc-access4` | — |
|
||||
| DC | Host | `dc-server1`-`dc-server4` | — |
|
||||
| Core | Core router | `core1`, `core2` | 65500 |
|
||||
| Campus | Spine | `campus-spine1`, `campus-spine2` | 66000 |
|
||||
| Campus | Leaf VTEP1 (MLAG) | `campus-leaf1`, `campus-leaf2` | 66001 |
|
||||
| Campus | Leaf VTEP2 (MLAG) | `campus-leaf3`, `campus-leaf4` | 66002 |
|
||||
| Campus | Border Leaf (MLAG) | `campus-border-leaf1`, `campus-border-leaf2` | 66005 |
|
||||
| Campus | Access (L2-only) | `campus-access1`, `campus-access2` | — |
|
||||
| Campus | Host | `campus-host1`, `campus-host2` | — |
|
||||
| Zone | Role | Nodes | AS |
|
||||
| ------ | ------------------ | -------------------------------------------- | ----- |
|
||||
| DC | Spine | `dc-spine1`, `dc-spine2` | 65000 |
|
||||
| DC | Leaf VTEP1 (MLAG) | `dc-leaf1`, `dc-leaf2` | 65001 |
|
||||
| DC | Leaf VTEP2 (MLAG) | `dc-leaf3`, `dc-leaf4` | 65002 |
|
||||
| DC | Leaf VTEP3 (MLAG) | `dc-leaf5`, `dc-leaf6` | 65003 |
|
||||
| DC | Leaf VTEP4 (MLAG) | `dc-leaf7`, `dc-leaf8` | 65004 |
|
||||
| DC | Border Leaf (MLAG) | `dc-border-leaf1`, `dc-border-leaf2` | 65005 |
|
||||
| DC | Access (L2-only) | `dc-access1`-`dc-access4` | — |
|
||||
| DC | Host | `dc-server1`-`dc-server4` | — |
|
||||
| Core | Core router | `core1`, `core2` | 65500 |
|
||||
| Campus | Spine | `campus-spine1`, `campus-spine2` | 66000 |
|
||||
| Campus | Leaf VTEP1 (MLAG) | `campus-leaf1`, `campus-leaf2` | 66001 |
|
||||
| Campus | Leaf VTEP2 (MLAG) | `campus-leaf3`, `campus-leaf4` | 66002 |
|
||||
| Campus | Border Leaf (MLAG) | `campus-border-leaf1`, `campus-border-leaf2` | 66005 |
|
||||
| Campus | Access (L2-only) | `campus-access1`, `campus-access2` | — |
|
||||
| Campus | Host | `campus-host1`, `campus-host2` | — |
|
||||
|
||||
### AS Numbering
|
||||
|
||||
| AS | Role |
|
||||
| ----- | ---------------------------------- |
|
||||
| 65000 | DC Spine |
|
||||
| 65001 | DC VTEP1 (dc-leaf1/2) |
|
||||
| 65002 | DC VTEP2 (dc-leaf3/4) |
|
||||
| 65003 | DC VTEP3 (dc-leaf5/6) |
|
||||
| 65004 | DC VTEP4 (dc-leaf7/8) |
|
||||
| 65005 | DC Border Leaf pair |
|
||||
| 65500 | Core (iBGP between core1 & core2) |
|
||||
| 66000 | Campus Spine |
|
||||
| 66001 | Campus VTEP1 (campus-leaf1/2) |
|
||||
| 66002 | Campus VTEP2 (campus-leaf3/4) |
|
||||
| 66005 | Campus Border Leaf pair |
|
||||
| AS | Role |
|
||||
| ----- | --------------------------------- |
|
||||
| 65000 | DC Spine |
|
||||
| 65001 | DC VTEP1 (dc-leaf1/2) |
|
||||
| 65002 | DC VTEP2 (dc-leaf3/4) |
|
||||
| 65003 | DC VTEP3 (dc-leaf5/6) |
|
||||
| 65004 | DC VTEP4 (dc-leaf7/8) |
|
||||
| 65005 | DC Border Leaf pair |
|
||||
| 65500 | Core (iBGP between core1 & core2) |
|
||||
| 66000 | Campus Spine |
|
||||
| 66001 | Campus VTEP1 (campus-leaf1/2) |
|
||||
| 66002 | Campus VTEP2 (campus-leaf3/4) |
|
||||
| 66005 | Campus Border Leaf pair |
|
||||
|
||||
### Access Switches
|
||||
|
||||
| Access Switch | Uplink Pair | VLANs | Host | Host attachment |
|
||||
| --------------- | ------------------------ | -------- | -------------- | ------------------------- |
|
||||
| dc-access1 | dc-leaf1/2 (VTEP1) | 40 | dc-server1 | LACP Po1 (dual-homed) |
|
||||
| dc-access2 | dc-leaf3/4 (VTEP2) | 34 | dc-server2 | LACP Po1 (dual-homed) |
|
||||
| dc-access3 | dc-leaf5/6 (VTEP3) | 40 | dc-server3 | LACP Po1 (dual-homed) |
|
||||
| dc-access4 | dc-leaf7/8 (VTEP4) | 78 | dc-server4 | LACP Po1 (dual-homed) |
|
||||
| campus-access1 | campus-leaf1/2 (VTEP1) | 60 | campus-host1 | access port (single link) |
|
||||
| campus-access2 | campus-leaf3/4 (VTEP2) | 70 | campus-host2 | access port (single link) |
|
||||
| Access Switch | Uplink Pair | VLANs | Host | Host attachment |
|
||||
| -------------- | ---------------------- | ----- | ------------ | ------------------------- |
|
||||
| dc-access1 | dc-leaf1/2 (VTEP1) | 40 | dc-server1 | LACP Po1 (dual-homed) |
|
||||
| dc-access2 | dc-leaf3/4 (VTEP2) | 34 | dc-server2 | LACP Po1 (dual-homed) |
|
||||
| dc-access3 | dc-leaf5/6 (VTEP3) | 40 | dc-server3 | LACP Po1 (dual-homed) |
|
||||
| dc-access4 | dc-leaf7/8 (VTEP4) | 78 | dc-server4 | LACP Po1 (dual-homed) |
|
||||
| campus-access1 | campus-leaf1/2 (VTEP1) | 60 | campus-host1 | access port (single link) |
|
||||
| campus-access2 | campus-leaf3/4 (VTEP2) | 70 | campus-host2 | access port (single link) |
|
||||
|
||||
All access switches are L2-only, LACP-bonded to their leaf MLAG pair via `Port-Channel10`. MSTP + edge-port BPDU guard.
|
||||
|
||||
@@ -122,71 +122,73 @@ Host-facing ports:
|
||||
|
||||
### Management (`172.16.0.0/24`)
|
||||
|
||||
| Node | IP | Node | IP |
|
||||
| ------------------------- | --------------- | ------------------------- | --------------- |
|
||||
| dc-spine1 | 172.16.0.1 | campus-spine1 | 172.16.0.20 |
|
||||
| dc-spine2 | 172.16.0.2 | campus-spine2 | 172.16.0.21 |
|
||||
| dc-border-leaf1 | 172.16.0.3 | campus-border-leaf1 | 172.16.0.22 |
|
||||
| dc-border-leaf2 | 172.16.0.4 | campus-border-leaf2 | 172.16.0.23 |
|
||||
| core1 | 172.16.0.10 | campus-leaf1-4 | 172.16.0.51-54 |
|
||||
| core2 | 172.16.0.11 | campus-access1 | 172.16.0.61 |
|
||||
| dc-leaf1 | 172.16.0.25 | campus-access2 | 172.16.0.62 |
|
||||
| dc-leaf2 | 172.16.0.50 | dc-server1-4 | 172.16.0.101-104|
|
||||
| dc-leaf3-8 | 172.16.0.27-32 | campus-host1 | 172.16.0.105 |
|
||||
| dc-access1-4 | 172.16.0.41-44 | campus-host2 | 172.16.0.106 |
|
||||
| Node | IP | Node | IP |
|
||||
| --------------- | -------------- | ------------------- | ---------------- |
|
||||
| dc-spine1 | 172.16.0.1 | campus-spine1 | 172.16.0.20 |
|
||||
| dc-spine2 | 172.16.0.2 | campus-spine2 | 172.16.0.21 |
|
||||
| dc-border-leaf1 | 172.16.0.3 | campus-border-leaf1 | 172.16.0.22 |
|
||||
| dc-border-leaf2 | 172.16.0.4 | campus-border-leaf2 | 172.16.0.23 |
|
||||
| core1 | 172.16.0.10 | campus-leaf1-4 | 172.16.0.51-54 |
|
||||
| core2 | 172.16.0.11 | campus-access1 | 172.16.0.61 |
|
||||
| dc-leaf1 | 172.16.0.25 | campus-access2 | 172.16.0.62 |
|
||||
| dc-leaf2 | 172.16.0.50 | dc-server1-4 | 172.16.0.101-104 |
|
||||
| dc-leaf3-8 | 172.16.0.27-32 | campus-host1 | 172.16.0.105 |
|
||||
| dc-access1-4 | 172.16.0.41-44 | campus-host2 | 172.16.0.106 |
|
||||
| | | gnmic | 172.16.0.70 |
|
||||
| | | prometheus | 172.16.0.71 |
|
||||
|
||||
Gateway: `172.16.0.254`.
|
||||
|
||||
### Router-ID Loopback0 (`Lo0`)
|
||||
|
||||
| Zone | Range | Nodes |
|
||||
| ------ | ------------------- | --------------------------------------------------------------------- |
|
||||
| DC | `10.0.250.0/24` | dc-spine1 .1, dc-spine2 .2, dc-leaf1-8 .11-.18, BL-dc1 .21, BL-dc2 .22 |
|
||||
| Core | `10.0.200.0/24` | core1 `10.0.200.1`, core2 `10.0.200.2` |
|
||||
| Campus | `10.1.250.0/24` | campus-spine1 .1, campus-spine2 .2, campus-leaf1-4 .11-.14, BL-campus1 .21, BL-campus2 .22 |
|
||||
| Zone | Range | Nodes |
|
||||
| ------ | --------------- | ------------------------------------------------------------------------------------------ |
|
||||
| DC | `10.0.250.0/24` | dc-spine1 .1, dc-spine2 .2, dc-leaf1-8 .11-.18, BL-dc1 .21, BL-dc2 .22 |
|
||||
| Core | `10.0.200.0/24` | core1 `10.0.200.1`, core2 `10.0.200.2` |
|
||||
| Campus | `10.1.250.0/24` | campus-spine1 .1, campus-spine2 .2, campus-leaf1-4 .11-.14, BL-campus1 .21, BL-campus2 .22 |
|
||||
|
||||
### VTEP Loopback1 (`Lo1`) — shared per MLAG pair
|
||||
|
||||
| Fabric | VTEP | Address | Leafs |
|
||||
| ------ | ------ | --------------- | ---------------------- |
|
||||
| DC | VTEP1 | `10.0.255.11` | dc-leaf1, dc-leaf2 |
|
||||
| DC | VTEP2 | `10.0.255.12` | dc-leaf3, dc-leaf4 |
|
||||
| DC | VTEP3 | `10.0.255.13` | dc-leaf5, dc-leaf6 |
|
||||
| DC | VTEP4 | `10.0.255.14` | dc-leaf7, dc-leaf8 |
|
||||
| DC | BL | `10.0.255.15` | dc-border-leaf1/2 |
|
||||
| Campus | VTEP1 | `10.1.255.11` | campus-leaf1/2 |
|
||||
| Campus | VTEP2 | `10.1.255.12` | campus-leaf3/4 |
|
||||
| Campus | BL | `10.1.255.21` | campus-border-leaf1/2 |
|
||||
| Fabric | VTEP | Address | Leafs |
|
||||
| ------ | ----- | ------------- | --------------------- |
|
||||
| DC | VTEP1 | `10.0.255.11` | dc-leaf1, dc-leaf2 |
|
||||
| DC | VTEP2 | `10.0.255.12` | dc-leaf3, dc-leaf4 |
|
||||
| DC | VTEP3 | `10.0.255.13` | dc-leaf5, dc-leaf6 |
|
||||
| DC | VTEP4 | `10.0.255.14` | dc-leaf7, dc-leaf8 |
|
||||
| DC | BL | `10.0.255.15` | dc-border-leaf1/2 |
|
||||
| Campus | VTEP1 | `10.1.255.11` | campus-leaf1/2 |
|
||||
| Campus | VTEP2 | `10.1.255.12` | campus-leaf3/4 |
|
||||
| Campus | BL | `10.1.255.21` | campus-border-leaf1/2 |
|
||||
|
||||
### Underlay P2P (`/31`)
|
||||
|
||||
| Segment | Subnets |
|
||||
| -------------------------------- | --------------------------------------- |
|
||||
| DC dc-spine1 ↔ leaf/BL | `10.0.1.0/31` … `10.0.1.18/31` |
|
||||
| DC dc-spine2 ↔ leaf/BL | `10.0.2.0/31` … `10.0.2.18/31` |
|
||||
| DC MLAG iBGP SVIs (per pair) | `10.0.3.0/31`, `.2/31`, `.4/31`, `.6/31`, `.8/31` (BL) |
|
||||
| DC MLAG peer-link SVIs | `10.0.199.240/31` … `10.0.199.246/31` |
|
||||
| DC-BL ↔ Core (default, `.100`) | `10.0.4.0/31` .. `10.0.4.6/31` |
|
||||
| DC-BL ↔ Core (VRF gold, `.200`) | `10.0.14.0/31` .. `10.0.14.6/31` |
|
||||
| Campus-BL ↔ Core (default) | `10.0.5.0/31` .. `10.0.5.6/31` |
|
||||
| Campus-BL ↔ Core (VRF gold) | `10.0.15.0/31` .. `10.0.15.6/31` |
|
||||
| Core1 ↔ Core2 (default) | `10.0.200.128/31` |
|
||||
| Core1 ↔ Core2 (VRF gold) | `10.0.200.130/31` |
|
||||
| Campus dc-spine1 ↔ leaf/BL | `10.1.1.0/31` … `10.1.1.10/31` |
|
||||
| Campus dc-spine2 ↔ leaf/BL | `10.1.2.0/31` … `10.1.2.10/31` |
|
||||
| Campus MLAG iBGP SVIs | `10.1.3.0/31`, `.2/31`, `.4/31` |
|
||||
| Campus MLAG peer-link SVIs | `10.1.199.250/31` … `10.1.199.254/31` |
|
||||
| Segment | Subnets |
|
||||
| ------------------------------- | ------------------------------------------------------ |
|
||||
| DC dc-spine1 ↔ leaf/BL | `10.0.1.0/31` … `10.0.1.18/31` |
|
||||
| DC dc-spine2 ↔ leaf/BL | `10.0.2.0/31` … `10.0.2.18/31` |
|
||||
| DC MLAG iBGP SVIs (per pair) | `10.0.3.0/31`, `.2/31`, `.4/31`, `.6/31`, `.8/31` (BL) |
|
||||
| DC MLAG peer-link SVIs | `10.0.199.240/31` … `10.0.199.246/31` |
|
||||
| DC-BL ↔ Core (default, `.100`) | `10.0.4.0/31` .. `10.0.4.6/31` |
|
||||
| DC-BL ↔ Core (VRF gold, `.200`) | `10.0.14.0/31` .. `10.0.14.6/31` |
|
||||
| Campus-BL ↔ Core (default) | `10.0.5.0/31` .. `10.0.5.6/31` |
|
||||
| Campus-BL ↔ Core (VRF gold) | `10.0.15.0/31` .. `10.0.15.6/31` |
|
||||
| Core1 ↔ Core2 (default) | `10.0.200.128/31` |
|
||||
| Core1 ↔ Core2 (VRF gold) | `10.0.200.130/31` |
|
||||
| Campus dc-spine1 ↔ leaf/BL | `10.1.1.0/31` … `10.1.1.10/31` |
|
||||
| Campus dc-spine2 ↔ leaf/BL | `10.1.2.0/31` … `10.1.2.10/31` |
|
||||
| Campus MLAG iBGP SVIs | `10.1.3.0/31`, `.2/31`, `.4/31` |
|
||||
| Campus MLAG peer-link SVIs | `10.1.199.250/31` … `10.1.199.254/31` |
|
||||
|
||||
### Host Addressing
|
||||
|
||||
| Host | VLAN | VRF | IP / Mask | Gateway | Purpose |
|
||||
| ------------- | ---- | -------- | ----------------- | ------------ | ------------------------------ |
|
||||
| dc-server1 | 40 | default | 10.40.40.101/24 | — | DC L2 stretched (VTEP1↔VTEP3) |
|
||||
| dc-server2 | 34 | gold | 10.34.34.102/24 | 10.34.34.1 | DC L3 VRF gold |
|
||||
| dc-server3 | 40 | default | 10.40.40.103/24 | — | DC L2 stretched |
|
||||
| dc-server4 | 78 | gold | 10.78.78.104/24 | 10.78.78.1 | DC L3 VRF gold |
|
||||
| campus-host1 | 60 | gold | 10.60.60.101/24 | 10.60.60.1 | Campus L3 VRF gold |
|
||||
| campus-host2 | 70 | gold | 10.60.70.102/24 | 10.60.70.1 | Campus L3 VRF gold |
|
||||
| Host | VLAN | VRF | IP / Mask | Gateway | Purpose |
|
||||
| ------------ | ---- | ------- | --------------- | ---------- | ----------------------------- |
|
||||
| dc-server1 | 40 | default | 10.40.40.101/24 | — | DC L2 stretched (VTEP1↔VTEP3) |
|
||||
| dc-server2 | 34 | gold | 10.34.34.102/24 | 10.34.34.1 | DC L3 VRF gold |
|
||||
| dc-server3 | 40 | default | 10.40.40.103/24 | — | DC L2 stretched |
|
||||
| dc-server4 | 78 | gold | 10.78.78.104/24 | 10.78.78.1 | DC L3 VRF gold |
|
||||
| campus-host1 | 60 | gold | 10.60.60.101/24 | 10.60.60.1 | Campus L3 VRF gold |
|
||||
| campus-host2 | 70 | gold | 10.60.70.102/24 | 10.60.70.1 | Campus L3 VRF gold |
|
||||
|
||||
> DC hosts are dual-homed in LACP over `bond0` with tagged VLAN sub-interfaces.
|
||||
> Campus hosts are single-attached with one untagged `eth1` in a single access VLAN.
|
||||
@@ -195,20 +197,20 @@ Gateway: `172.16.0.254`.
|
||||
|
||||
### L2 VNI Mapping
|
||||
|
||||
| VLAN | Description | VNI | Scope | RT |
|
||||
| ---- | ------------------------------ | ------ | ------------------------------------------------------ | ---------- |
|
||||
| 40 | DC L2 VXLAN (stretched) | 110040 | DC VTEP1 (dc-leaf1/2) + VTEP3 (dc-leaf5/6) | 40:110040 |
|
||||
| 50 | Campus L2 VXLAN (stretched) | 110050 | Campus VTEP1 (campus-leaf1/2) + VTEP2 (campus-leaf3/4) | 50:110050 |
|
||||
| 34 | DC VRF gold subnet (local) | 110034 | DC VTEP2 only (anycast GW 10.34.34.1) | 34:110034 |
|
||||
| 78 | DC VRF gold subnet (local) | 110078 | DC VTEP4 only (anycast GW 10.78.78.1) | 78:110078 |
|
||||
| 60 | Campus VRF gold subnet (local) | 110060 | Campus VTEP1 only (anycast GW 10.60.60.1) | 60:110060 |
|
||||
| 70 | Campus VRF gold subnet (local) | 110070 | Campus VTEP2 only (anycast GW 10.60.70.1) | 70:110070 |
|
||||
| VLAN | Description | VNI | Scope | RT |
|
||||
| ---- | ------------------------------ | ------ | ------------------------------------------------------ | --------- |
|
||||
| 40 | DC L2 VXLAN (stretched) | 110040 | DC VTEP1 (dc-leaf1/2) + VTEP3 (dc-leaf5/6) | 40:110040 |
|
||||
| 50 | Campus L2 VXLAN (stretched) | 110050 | Campus VTEP1 (campus-leaf1/2) + VTEP2 (campus-leaf3/4) | 50:110050 |
|
||||
| 34 | DC VRF gold subnet (local) | 110034 | DC VTEP2 only (anycast GW 10.34.34.1) | 34:110034 |
|
||||
| 78 | DC VRF gold subnet (local) | 110078 | DC VTEP4 only (anycast GW 10.78.78.1) | 78:110078 |
|
||||
| 60 | Campus VRF gold subnet (local) | 110060 | Campus VTEP1 only (anycast GW 10.60.60.1) | 60:110060 |
|
||||
| 70 | Campus VRF gold subnet (local) | 110070 | Campus VTEP2 only (anycast GW 10.60.70.1) | 70:110070 |
|
||||
|
||||
### L3 VNI Mapping (end-to-end)
|
||||
|
||||
| VRF | L3 VNI | RT | Scope |
|
||||
| ---- | ------- | ---------- | ----------------------------------------------------- |
|
||||
| gold | 100001 | 1:100001 | DC VTEP2/VTEP4/DC-BL + Campus VTEP1/VTEP2/Campus-BL |
|
||||
| VRF | L3 VNI | RT | Scope |
|
||||
| ---- | ------ | -------- | --------------------------------------------------- |
|
||||
| gold | 100001 | 1:100001 | DC VTEP2/VTEP4/DC-BL + Campus VTEP1/VTEP2/Campus-BL |
|
||||
|
||||
VRF `gold` is announced over EVPN Type-5 (IP prefix) inside each fabric, and **stitched by the Core** via eBGP IPv4 unicast in VRF gold (over the `.200` dot1q subinterfaces). L3 VNI `100001` is re-used end-to-end for symmetry; RT `1:100001` is consistent across both fabrics.
|
||||
|
||||
@@ -219,19 +221,19 @@ VRF `gold` is announced over EVPN Type-5 (IP prefix) inside each fabric, and **s
|
||||
|
||||
## 🔀 Control Plane Summary
|
||||
|
||||
| Segment | Protocol | Notes |
|
||||
| ----------------------------------- | ------------------------------------ | ------------------------------------- |
|
||||
| DC spine ↔ leaf/BL underlay | eBGP IPv4 (AS 65000 ↔ 650xx) | `maximum-paths 4 ecmp 64` |
|
||||
| DC spine ↔ leaf/BL overlay | eBGP EVPN via Loopback0, multi-hop 3 | Spines reflect via `ebgp peer-group` |
|
||||
| DC MLAG pair iBGP | iBGP over VLAN 4091 SVI | `next-hop-self` |
|
||||
| DC-BL ↔ Core (default) | OSPF area 0 + eBGP AS 65005 ↔ 65500 | on `.100` dot1q subinterface |
|
||||
| DC-BL ↔ Core (VRF gold) | eBGP AS 65005 ↔ 65500 | on `.200` dot1q subinterface |
|
||||
| Core1 ↔ Core2 (default) | OSPF area 0 + iBGP AS 65500 | via Loopback0 |
|
||||
| Core1 ↔ Core2 (VRF gold) | iBGP AS 65500 | VRF-aware over `.200` subinterface |
|
||||
| Campus-BL ↔ Core (default / gold) | OSPF + eBGP AS 66005 ↔ 65500 | same pattern as DC-BL |
|
||||
| Campus spine ↔ leaf/BL underlay | eBGP IPv4 (AS 66000 ↔ 660xx) | |
|
||||
| Campus spine ↔ leaf/BL overlay | eBGP EVPN via Loopback0, multi-hop 3 | |
|
||||
| Campus MLAG pair iBGP | iBGP over VLAN 4091 SVI | |
|
||||
| Segment | Protocol | Notes |
|
||||
| --------------------------------- | ------------------------------------ | ------------------------------------ |
|
||||
| DC spine ↔ leaf/BL underlay | eBGP IPv4 (AS 65000 ↔ 650xx) | `maximum-paths 4 ecmp 64` |
|
||||
| DC spine ↔ leaf/BL overlay | eBGP EVPN via Loopback0, multi-hop 3 | Spines reflect via `ebgp peer-group` |
|
||||
| DC MLAG pair iBGP | iBGP over VLAN 4091 SVI | `next-hop-self` |
|
||||
| DC-BL ↔ Core (default) | OSPF area 0 + eBGP AS 65005 ↔ 65500 | on `.100` dot1q subinterface |
|
||||
| DC-BL ↔ Core (VRF gold) | eBGP AS 65005 ↔ 65500 | on `.200` dot1q subinterface |
|
||||
| Core1 ↔ Core2 (default) | OSPF area 0 + iBGP AS 65500 | via Loopback0 |
|
||||
| Core1 ↔ Core2 (VRF gold) | iBGP AS 65500 | VRF-aware over `.200` subinterface |
|
||||
| Campus-BL ↔ Core (default / gold) | OSPF + eBGP AS 66005 ↔ 65500 | same pattern as DC-BL |
|
||||
| Campus spine ↔ leaf/BL underlay | eBGP IPv4 (AS 66000 ↔ 660xx) | |
|
||||
| Campus spine ↔ leaf/BL overlay | eBGP EVPN via Loopback0, multi-hop 3 | |
|
||||
| Campus MLAG pair iBGP | iBGP over VLAN 4091 SVI | |
|
||||
|
||||
## 🧪 Testing & Validation
|
||||
|
||||
@@ -323,6 +325,74 @@ ssh admin@clab-arista-evpn-fabric-dc-border-leaf1 "show bgp evpn route-type ip-p
|
||||
ssh admin@clab-arista-evpn-fabric-campus-border-leaf1 "show bgp evpn route-type ip-prefix ipv4"
|
||||
```
|
||||
|
||||
## 📡 Telemetry
|
||||
|
||||
A [`gnmic`](https://gnmic.openconfig.net/) container node subscribes to gNMI (port `6030`,
|
||||
`eos-native` provider) on every Arista `cEOS` node in the fabric, and exposes the
|
||||
collected metrics on a Prometheus exporter endpoint.
|
||||
|
||||
Subscriptions (see issue #44 for the path-selection rationale):
|
||||
|
||||
| Subscription | Nodes | Paths |
|
||||
| -------------- | -------------------------------------------------- | ---------------------------------------------------------------------------------------- |
|
||||
| `eos-interfaces` | all | `interfaces/interface/state/{oper-status,counters}` |
|
||||
| `eos-bgp` | all | `.../bgp/neighbors/neighbor/state/session-state`, `.../afi-safis/afi-safi/state/prefixes` (IPv4/IPv6 unicast + L2VPN EVPN) |
|
||||
| `eos-system` | all | `/system/cpus/cpu/state`, `/system/memory/state` |
|
||||
| `eos-vxlan` | VTEP nodes only (leafs + border-leafs, both fabrics) | VLAN-to-VNI mapping + MAC table entries — joined in Prometheus to get MAC count per VNI |
|
||||
|
||||
- Config: `configs/gnmic/gnmic-config.yml`
|
||||
- TLS: `insecure: true` — cEOS gNMI runs in plaintext here (no SSL profile configured on
|
||||
`management api gnmi`), not TLS; `skip-verify` would fail the handshake
|
||||
- `oper-status` and BGP `session-state` are string enums; gnmic's Prometheus output drops
|
||||
non-numeric leaves, so the `state-to-int` event-processor maps them to `1`/`0`
|
||||
- Prometheus exporter: `http://clab-arista-evpn-fabric-gnmic:9273/metrics`
|
||||
|
||||
A `prometheus` container node (`prom/prometheus`, `172.16.0.71:9090`) scrapes the
|
||||
gnmic exporter every `5s` and is deployed inside the topology (`clab deploy`/`clab
|
||||
destroy`, self-contained — see issue #45). Metric names are kept as-is; raw
|
||||
OpenConfig-flattened labels are relabeled to clean, joinable names:
|
||||
|
||||
| Raw label | Clean label |
|
||||
| -------------------------------- | ------------------ |
|
||||
| `source` | `device` |
|
||||
| `interface_name` | `interface` |
|
||||
| `neighbor_neighbor_address` | `neighbor_address` |
|
||||
| `vlan_to_vni_vlan` / `entry_vlan` | `vlan` |
|
||||
| `entry_mac_address` | `mac_address` |
|
||||
| `afi_safi_afi_safi_name` | `afi_safi` |
|
||||
|
||||
`vni` is **not** available as a native label on any gnmic-exported metric — it
|
||||
only appears as the sample *value* of
|
||||
`interfaces_interface_arista_vxlan_vlan_to_vnis_vlan_to_vni_state_vni`, keyed by
|
||||
`(device, interface, vlan)`. Per-VNI MAC counts require a PromQL join on `vlan`
|
||||
rather than a native `vni` label.
|
||||
|
||||
`site` (`dc`/`campus`/`core`) is also not gnmic-exported. It's added as a
|
||||
Prometheus `metric_relabel_configs` rule deriving it from `device` via the
|
||||
`<area>-<role><n>` naming convention regex (see #49), rather than repeating
|
||||
`label_replace()` in every dashboard query.
|
||||
|
||||
- Config: `configs/prometheus/prometheus.yml`
|
||||
- This is separate from, and does not replace, the existing external Prometheus
|
||||
instance — no cutover yet, both run in parallel pending validation
|
||||
|
||||
```bash
|
||||
# Validate gnmic is subscribed and streaming from all targets
|
||||
docker logs clab-arista-evpn-fabric-gnmic
|
||||
|
||||
# Scrape the Prometheus exporter directly
|
||||
docker exec clab-arista-evpn-fabric-gnmic wget -qO- http://localhost:9273/metrics | head
|
||||
|
||||
# Query the in-topology Prometheus instance
|
||||
curl -s 'http://172.16.0.71:9090/api/v1/query?query=system_memory_state_used' | jq
|
||||
```
|
||||
|
||||
### Weathermap panel generation (dashboard-as-code)
|
||||
|
||||
`scripts/generate_weathermap.py` generates a weathermap-ng Grafana panel from
|
||||
live IPFabric topology + gnmic/Prometheus metrics. See
|
||||
[`scripts/README.md`](scripts/README.md) for usage and details.
|
||||
|
||||
## 📁 Repository Structure
|
||||
|
||||
```
|
||||
@@ -343,7 +413,16 @@ arista-evpn-vxlan-clab/
|
||||
│ ├── campus-spine1.cfg, campus-spine2.cfg
|
||||
│ ├── campus-leaf1.cfg … campus-leaf4.cfg
|
||||
│ ├── campus-border-leaf1.cfg, campus-border-leaf2.cfg
|
||||
│ └── campus-access1.cfg, campus-access2.cfg
|
||||
│ ├── campus-access1.cfg, campus-access2.cfg
|
||||
│ ├── gnmic/
|
||||
│ │ └── gnmic-config.yml
|
||||
│ ├── prometheus/
|
||||
│ │ └── prometheus.yml
|
||||
│ └── grafana/
|
||||
│ └── weathermap-dashboard.json
|
||||
├── scripts/
|
||||
│ ├── README.md
|
||||
│ └── generate_weathermap.py
|
||||
└── hosts/
|
||||
├── README.md
|
||||
├── dc-server1_interfaces … dc-server4_interfaces
|
||||
|
||||
BIN
assets/grafana-weathermap-dashboard.png
Normal file
BIN
assets/grafana-weathermap-dashboard.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 639 KiB |
72
assets/weathermap-pipeline.svg
Normal file
72
assets/weathermap-pipeline.svg
Normal file
@@ -0,0 +1,72 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 900 350" font-family="Helvetica, Arial, sans-serif">
|
||||
<defs>
|
||||
<marker id="arrow" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse">
|
||||
<path d="M 0 0 L 10 5 L 0 10 z" fill="#555"/>
|
||||
</marker>
|
||||
</defs>
|
||||
|
||||
<style>
|
||||
.box { stroke-width: 1.5; rx: 6; }
|
||||
.input { fill: #e8f0fe; stroke: #4285f4; }
|
||||
.manual { fill: #fef7e0; stroke: #f9ab00; }
|
||||
.script { fill: #e6f4ea; stroke: #34a853; }
|
||||
.output { fill: #f3e8fd; stroke: #a142f4; }
|
||||
.grafana { fill: #fce8e6; stroke: #ea4335; }
|
||||
.label { font-size: 13px; fill: #202124; }
|
||||
.sublabel { font-size: 10px; fill: #5f6368; }
|
||||
.arrow { stroke: #555; stroke-width: 1.5; fill: none; marker-end: url(#arrow); }
|
||||
.caption { font-size: 11px; fill: #5f6368; font-style: italic; }
|
||||
</style>
|
||||
|
||||
<!-- inputs -->
|
||||
<rect class="box input" x="20" y="30" width="150" height="50"/>
|
||||
<text class="label" x="95" y="50" text-anchor="middle">IPFabric</text>
|
||||
<text class="sublabel" x="95" y="66" text-anchor="middle">topology, inventory</text>
|
||||
|
||||
<rect class="box input" x="20" y="100" width="150" height="50"/>
|
||||
<text class="label" x="95" y="120" text-anchor="middle">Prometheus</text>
|
||||
<text class="sublabel" x="95" y="136" text-anchor="middle">gnmic metrics, live labels</text>
|
||||
|
||||
<!-- generator script -->
|
||||
<rect class="box script" x="220" y="55" width="200" height="70"/>
|
||||
<text class="label" x="320" y="82" text-anchor="middle" font-size="11.5">generate_weathermap.py</text>
|
||||
<text class="sublabel" x="320" y="98" text-anchor="middle">weathermap panel only</text>
|
||||
<text class="sublabel" x="320" y="112" text-anchor="middle">targets + options.weathermap</text>
|
||||
|
||||
<path class="arrow" d="M170,55 L220,80"/>
|
||||
<path class="arrow" d="M170,125 L220,100"/>
|
||||
|
||||
<!-- manual base -->
|
||||
<rect class="box manual" x="220" y="220" width="200" height="70"/>
|
||||
<text class="label" x="320" y="247" text-anchor="middle">dashboard-base.json</text>
|
||||
<text class="sublabel" x="320" y="263" text-anchor="middle">hand-authored, committed</text>
|
||||
<text class="sublabel" x="320" y="277" text-anchor="middle">BGP/ports/throughput/vars</text>
|
||||
|
||||
<!-- merge -->
|
||||
<rect class="box script" x="470" y="130" width="160" height="60"/>
|
||||
<text class="label" x="550" y="155" text-anchor="middle">merge</text>
|
||||
<text class="sublabel" x="550" y="171" text-anchor="middle" font-size="8.5">splices into __WEATHERMAP_SLOT__</text>
|
||||
|
||||
<path class="arrow" d="M420,90 L470,145"/>
|
||||
<path class="arrow" d="M420,255 L470,175"/>
|
||||
|
||||
<!-- output artifact -->
|
||||
<rect class="box output" x="660" y="60" width="210" height="60"/>
|
||||
<text class="label" x="765" y="85" text-anchor="middle" font-size="11.5">weathermap-dashboard.json</text>
|
||||
<text class="sublabel" x="765" y="101" text-anchor="middle">generated build artifact</text>
|
||||
|
||||
<path class="arrow" d="M630,150 L765,120"/>
|
||||
|
||||
<!-- grafana -->
|
||||
<rect class="box grafana" x="660" y="210" width="210" height="60"/>
|
||||
<text class="label" x="765" y="235" text-anchor="middle">Grafana</text>
|
||||
<text class="sublabel" x="765" y="251" text-anchor="middle" font-size="9">--provision (POST /api/dashboards/db)</text>
|
||||
|
||||
<path class="arrow" d="M630,175 L765,210"/>
|
||||
<path class="arrow" d="M765,120 L765,210" stroke-dasharray="3,3"/>
|
||||
|
||||
<!-- read-back for position preservation -->
|
||||
<path class="arrow" d="M765,270 C 620,330 490,300 420,275" stroke-dasharray="3,3"/>
|
||||
<text class="caption" x="545" y="325" text-anchor="middle">live node positions read back before each regeneration (see #53)</text>
|
||||
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 3.5 KiB |
135
configs/gnmic/gnmic-config.yml
Normal file
135
configs/gnmic/gnmic-config.yml
Normal file
@@ -0,0 +1,135 @@
|
||||
username: admin
|
||||
password: admin
|
||||
port: 6030
|
||||
timeout: 10s
|
||||
|
||||
common_eos_subscriptions: &common_eos_subs
|
||||
insecure: true
|
||||
subscriptions:
|
||||
- eos-interfaces
|
||||
- eos-bgp
|
||||
- eos-system
|
||||
|
||||
vtep_eos_subscriptions: &vtep_eos_subs
|
||||
insecure: true
|
||||
subscriptions:
|
||||
- eos-interfaces
|
||||
- eos-bgp
|
||||
- eos-system
|
||||
- eos-vxlan
|
||||
|
||||
targets:
|
||||
# DC fabric
|
||||
dc-spine1: *common_eos_subs
|
||||
dc-spine2: *common_eos_subs
|
||||
dc-leaf1: *vtep_eos_subs
|
||||
dc-leaf2: *vtep_eos_subs
|
||||
dc-leaf3: *vtep_eos_subs
|
||||
dc-leaf4: *vtep_eos_subs
|
||||
dc-leaf5: *vtep_eos_subs
|
||||
dc-leaf6: *vtep_eos_subs
|
||||
dc-leaf7: *vtep_eos_subs
|
||||
dc-leaf8: *vtep_eos_subs
|
||||
dc-border-leaf1: *vtep_eos_subs
|
||||
dc-border-leaf2: *vtep_eos_subs
|
||||
dc-access1: *common_eos_subs
|
||||
dc-access2: *common_eos_subs
|
||||
dc-access3: *common_eos_subs
|
||||
dc-access4: *common_eos_subs
|
||||
|
||||
# Core
|
||||
core1: *common_eos_subs
|
||||
core2: *common_eos_subs
|
||||
|
||||
# Campus fabric
|
||||
campus-spine1: *common_eos_subs
|
||||
campus-spine2: *common_eos_subs
|
||||
campus-border-leaf1: *vtep_eos_subs
|
||||
campus-border-leaf2: *vtep_eos_subs
|
||||
campus-leaf1: *vtep_eos_subs
|
||||
campus-leaf2: *vtep_eos_subs
|
||||
campus-leaf3: *vtep_eos_subs
|
||||
campus-leaf4: *vtep_eos_subs
|
||||
campus-access1: *common_eos_subs
|
||||
campus-access2: *common_eos_subs
|
||||
|
||||
subscriptions:
|
||||
eos-interfaces:
|
||||
mode: stream
|
||||
stream-mode: sample
|
||||
sample-interval: 10s
|
||||
paths:
|
||||
- /interfaces/interface/state/oper-status
|
||||
- /interfaces/interface/state/counters
|
||||
eos-bgp:
|
||||
mode: stream
|
||||
stream-mode: sample
|
||||
sample-interval: 10s
|
||||
paths:
|
||||
- /network-instances/network-instance/protocols/protocol/bgp/neighbors/neighbor/state/session-state
|
||||
- /network-instances/network-instance/protocols/protocol/bgp/neighbors/neighbor/afi-safis/afi-safi/state/prefixes
|
||||
eos-system:
|
||||
mode: stream
|
||||
stream-mode: sample
|
||||
sample-interval: 10s
|
||||
paths:
|
||||
- /system/cpus/cpu/state
|
||||
- /system/memory/state
|
||||
eos-vxlan:
|
||||
mode: stream
|
||||
stream-mode: sample
|
||||
sample-interval: 10s
|
||||
paths:
|
||||
# VLAN-to-VNI mapping (join key for MAC-count-per-VNI in Prometheus)
|
||||
- /interfaces/interface[name=Vxlan1]/arista-exp-eos-vxlan:arista-vxlan/vlan-to-vnis/vlan-to-vni
|
||||
# MAC table entries, tagged by VLAN — joined with the mapping above to get per-VNI counts
|
||||
- /network-instances/network-instance/fdb/mac-table/entries/entry
|
||||
|
||||
outputs:
|
||||
prom-output:
|
||||
type: prometheus
|
||||
listen: :9273
|
||||
event-processors:
|
||||
- state-to-int
|
||||
|
||||
# oper-status and BGP session-state are string enums; the Prometheus output
|
||||
# silently drops non-numeric values, so they're mapped to 1 (up/established) / 0 (anything else)
|
||||
processors:
|
||||
state-to-int:
|
||||
event-strings:
|
||||
value-names:
|
||||
- ".*oper-status$"
|
||||
- ".*session-state$"
|
||||
transforms:
|
||||
- replace:
|
||||
apply-on: "value"
|
||||
old: "UP"
|
||||
new: "1"
|
||||
- replace:
|
||||
apply-on: "value"
|
||||
old: "ESTABLISHED"
|
||||
new: "1"
|
||||
- replace:
|
||||
apply-on: "value"
|
||||
old: "DOWN"
|
||||
new: "0"
|
||||
- replace:
|
||||
apply-on: "value"
|
||||
old: "IDLE"
|
||||
new: "0"
|
||||
- replace:
|
||||
apply-on: "value"
|
||||
old: "CONNECT"
|
||||
new: "0"
|
||||
- replace:
|
||||
apply-on: "value"
|
||||
old: "ACTIVE"
|
||||
new: "0"
|
||||
- replace:
|
||||
apply-on: "value"
|
||||
old: "OPENSENT"
|
||||
new: "0"
|
||||
- replace:
|
||||
apply-on: "value"
|
||||
old: "OPENCONFIRM"
|
||||
new: "0"
|
||||
197
configs/grafana/dashboard-base.json
Normal file
197
configs/grafana/dashboard-base.json
Normal file
@@ -0,0 +1,197 @@
|
||||
{
|
||||
"uid": "evpn-vxlan-fabric-weathermap",
|
||||
"title": "EVPN/VXLAN Fabric Weathermap",
|
||||
"tags": ["evpn", "vxlan", "weathermap", "auto-generated"],
|
||||
"timezone": "browser",
|
||||
"schemaVersion": 39,
|
||||
"version": 0,
|
||||
"templating": {
|
||||
"list": [
|
||||
{
|
||||
"name": "site",
|
||||
"type": "query",
|
||||
"datasource": { "type": "prometheus", "uid": "__DATASOURCE_UID__" },
|
||||
"query": { "query": "label_values(interfaces_interface_state_oper_status, site)", "refId": "site" },
|
||||
"refresh": 2,
|
||||
"sort": 1,
|
||||
"multi": true,
|
||||
"includeAll": true,
|
||||
"current": { "text": "All", "value": "$__all" }
|
||||
},
|
||||
{
|
||||
"name": "device",
|
||||
"type": "query",
|
||||
"datasource": { "type": "prometheus", "uid": "__DATASOURCE_UID__" },
|
||||
"query": {
|
||||
"query": "label_values(interfaces_interface_state_oper_status{site=~\"$site\"}, device)",
|
||||
"refId": "device"
|
||||
},
|
||||
"refresh": 2,
|
||||
"sort": 1,
|
||||
"multi": true,
|
||||
"includeAll": true,
|
||||
"current": { "text": "All", "value": "$__all" }
|
||||
}
|
||||
]
|
||||
},
|
||||
"panels": [
|
||||
{
|
||||
"id": 1,
|
||||
"title": "__WEATHERMAP_SLOT__",
|
||||
"gridPos": { "h": 24, "w": 24, "x": 0, "y": 0 }
|
||||
},
|
||||
{
|
||||
"id": 2,
|
||||
"type": "table",
|
||||
"title": "BGP Sessions",
|
||||
"gridPos": { "h": 10, "w": 12, "x": 0, "y": 24 },
|
||||
"datasource": { "type": "prometheus", "uid": "__DATASOURCE_UID__" },
|
||||
"targets": [
|
||||
{
|
||||
"refId": "A",
|
||||
"datasource": { "type": "prometheus", "uid": "__DATASOURCE_UID__" },
|
||||
"instant": true,
|
||||
"format": "table",
|
||||
"expr": "network_instances_network_instance_protocols_protocol_bgp_neighbors_neighbor_state_session_state{device=~\"$device\"}"
|
||||
}
|
||||
],
|
||||
"transformations": [
|
||||
{
|
||||
"id": "organize",
|
||||
"options": {
|
||||
"excludeByName": {
|
||||
"Time": true,
|
||||
"__name__": true,
|
||||
"instance": true,
|
||||
"job": true,
|
||||
"protocol_identifier": true,
|
||||
"protocol_name": true,
|
||||
"subscription_name": true
|
||||
},
|
||||
"renameByName": {
|
||||
"device": "Device",
|
||||
"neighbor_address": "Neighbor Address",
|
||||
"network_instance_name": "VRF",
|
||||
"Value": "State"
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"mappings": [
|
||||
{ "type": "value", "options": { "0": { "text": "Down", "color": "red" }, "1": { "text": "Up", "color": "green" } } }
|
||||
],
|
||||
"custom": { "cellOptions": { "type": "color-background" } }
|
||||
},
|
||||
"overrides": [
|
||||
{
|
||||
"matcher": { "id": "byName", "options": "State" },
|
||||
"properties": [
|
||||
{ "id": "custom.cellOptions", "value": { "type": "color-background" } },
|
||||
{
|
||||
"id": "mappings",
|
||||
"value": [
|
||||
{ "type": "value", "options": { "0": { "text": "Down", "color": "red" }, "1": { "text": "Up", "color": "green" } } }
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": 3,
|
||||
"type": "table",
|
||||
"title": "Ports / Interfaces",
|
||||
"gridPos": { "h": 10, "w": 12, "x": 12, "y": 24 },
|
||||
"datasource": { "type": "prometheus", "uid": "__DATASOURCE_UID__" },
|
||||
"targets": [
|
||||
{
|
||||
"refId": "A",
|
||||
"datasource": { "type": "prometheus", "uid": "__DATASOURCE_UID__" },
|
||||
"instant": true,
|
||||
"format": "table",
|
||||
"expr": "interfaces_interface_state_oper_status{device=~\"$device\"}"
|
||||
},
|
||||
{
|
||||
"refId": "B",
|
||||
"datasource": { "type": "prometheus", "uid": "__DATASOURCE_UID__" },
|
||||
"instant": true,
|
||||
"format": "table",
|
||||
"expr": "rate(interfaces_interface_state_counters_in_octets{device=~\"$device\"}[5m]) * 8"
|
||||
},
|
||||
{
|
||||
"refId": "C",
|
||||
"datasource": { "type": "prometheus", "uid": "__DATASOURCE_UID__" },
|
||||
"instant": true,
|
||||
"format": "table",
|
||||
"expr": "rate(interfaces_interface_state_counters_out_octets{device=~\"$device\"}[5m]) * 8"
|
||||
}
|
||||
],
|
||||
"transformations": [
|
||||
{ "id": "merge", "options": {} },
|
||||
{
|
||||
"id": "organize",
|
||||
"options": {
|
||||
"excludeByName": {
|
||||
"Time": true,
|
||||
"__name__": true,
|
||||
"instance": true,
|
||||
"job": true,
|
||||
"subscription_name": true
|
||||
},
|
||||
"renameByName": {
|
||||
"device": "Device",
|
||||
"interface": "Interface",
|
||||
"Value #A": "Status",
|
||||
"Value #B": "In (bps)",
|
||||
"Value #C": "Out (bps)"
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"fieldConfig": {
|
||||
"defaults": {},
|
||||
"overrides": [
|
||||
{
|
||||
"matcher": { "id": "byName", "options": "Status" },
|
||||
"properties": [
|
||||
{ "id": "custom.cellOptions", "value": { "type": "color-background" } },
|
||||
{
|
||||
"id": "mappings",
|
||||
"value": [
|
||||
{ "type": "value", "options": { "0": { "text": "Down", "color": "red" }, "1": { "text": "Up", "color": "green" } } }
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
{ "matcher": { "id": "byName", "options": "In (bps)" }, "properties": [{ "id": "unit", "value": "bps" }] },
|
||||
{ "matcher": { "id": "byName", "options": "Out (bps)" }, "properties": [{ "id": "unit", "value": "bps" }] }
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": 4,
|
||||
"type": "timeseries",
|
||||
"title": "Throughput per Site",
|
||||
"gridPos": { "h": 10, "w": 24, "x": 0, "y": 34 },
|
||||
"datasource": { "type": "prometheus", "uid": "__DATASOURCE_UID__" },
|
||||
"fieldConfig": { "defaults": { "unit": "bps" }, "overrides": [] },
|
||||
"targets": [
|
||||
{
|
||||
"refId": "A",
|
||||
"datasource": { "type": "prometheus", "uid": "__DATASOURCE_UID__" },
|
||||
"expr": "sum by (site) (rate(interfaces_interface_state_counters_in_octets{site=~\"$site\", interface!~\"Management.*\"}[5m]) * 8)",
|
||||
"legendFormat": "{{site}} in"
|
||||
},
|
||||
{
|
||||
"refId": "B",
|
||||
"datasource": { "type": "prometheus", "uid": "__DATASOURCE_UID__" },
|
||||
"expr": "-sum by (site) (rate(interfaces_interface_state_counters_out_octets{site=~\"$site\", interface!~\"Management.*\"}[5m]) * 8)",
|
||||
"legendFormat": "{{site}} out"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
4233
configs/grafana/weathermap-dashboard.json
Normal file
4233
configs/grafana/weathermap-dashboard.json
Normal file
File diff suppressed because it is too large
Load Diff
92
configs/prometheus/prometheus.yml
Normal file
92
configs/prometheus/prometheus.yml
Normal file
@@ -0,0 +1,92 @@
|
||||
global:
|
||||
scrape_interval: 5s
|
||||
evaluation_interval: 5s
|
||||
|
||||
scrape_configs:
|
||||
- job_name: gnmic
|
||||
static_configs:
|
||||
- targets: ["172.16.0.70:9273"]
|
||||
metric_relabel_configs:
|
||||
# device: raw label is "source" (gnmic subscription target)
|
||||
- source_labels: [source]
|
||||
target_label: device
|
||||
regex: (.+)
|
||||
replacement: $1
|
||||
action: replace
|
||||
- action: labeldrop
|
||||
regex: source
|
||||
|
||||
# site: not exported by gnmic -- derived from the hostname naming
|
||||
# convention (see CLAUDE.md: dc-*/campus-*/core* regex used for
|
||||
# IPFabric site separation) here instead of repeating label_replace()
|
||||
# in every dashboard query, see issue #49
|
||||
- source_labels: [device]
|
||||
target_label: site
|
||||
regex: (campus|core|dc)-?.*
|
||||
replacement: $1
|
||||
action: replace
|
||||
|
||||
# interface: raw label is already "interface_name" (OpenConfig key on the
|
||||
# "interface" path element) — normalize to "interface" for consistency
|
||||
- source_labels: [interface_name]
|
||||
target_label: interface
|
||||
regex: (.+)
|
||||
replacement: $1
|
||||
action: replace
|
||||
- action: labeldrop
|
||||
regex: interface_name
|
||||
|
||||
# BGP neighbor address: raw label is double-prefixed "neighbor_neighbor_address"
|
||||
# (OpenConfig path flattening — key name matches the path element name),
|
||||
# see issue #44
|
||||
- source_labels: [neighbor_neighbor_address]
|
||||
target_label: neighbor_address
|
||||
regex: (.+)
|
||||
replacement: $1
|
||||
action: replace
|
||||
- action: labeldrop
|
||||
regex: neighbor_neighbor_address
|
||||
|
||||
# VLAN id: normalize the two raw spellings ("vlan_to_vni_vlan" on the
|
||||
# VLAN-to-VNI mapping, "entry_vlan" on FDB entries) to a single "vlan" label
|
||||
- source_labels: [vlan_to_vni_vlan]
|
||||
target_label: vlan
|
||||
regex: (.+)
|
||||
replacement: $1
|
||||
action: replace
|
||||
- action: labeldrop
|
||||
regex: vlan_to_vni_vlan
|
||||
- source_labels: [entry_vlan]
|
||||
target_label: vlan
|
||||
regex: (.+)
|
||||
replacement: $1
|
||||
action: replace
|
||||
- action: labeldrop
|
||||
regex: entry_vlan
|
||||
|
||||
# NOTE: "vni" is NOT available as a label from any gnmic-exported metric.
|
||||
# The VNI only appears as the sample *value* of
|
||||
# interfaces_interface_arista_vxlan_vlan_to_vnis_vlan_to_vni_state_vni,
|
||||
# keyed by (device, interface, vlan). Relabeling can only rename existing
|
||||
# labels, not promote a value to a label. Per-VNI MAC counts therefore
|
||||
# require a PromQL join (vlan_to_vni_state_vni joined with the FDB
|
||||
# entry count, grouped by vlan) rather than a native "vni" label — see
|
||||
# issue #45 for the documented gap.
|
||||
|
||||
# MAC address on FDB entries: drop the "entry_" prefix for consistency
|
||||
- source_labels: [entry_mac_address]
|
||||
target_label: mac_address
|
||||
regex: (.+)
|
||||
replacement: $1
|
||||
action: replace
|
||||
- action: labeldrop
|
||||
regex: entry_mac_address
|
||||
|
||||
# AFI/SAFI on BGP prefix counters: drop the flattened prefix
|
||||
- source_labels: [afi_safi_afi_safi_name]
|
||||
target_label: afi_safi
|
||||
regex: (.+)
|
||||
replacement: $1
|
||||
action: replace
|
||||
- action: labeldrop
|
||||
regex: afi_safi_afi_safi_name
|
||||
11
envrc.sample
Normal file
11
envrc.sample
Normal file
@@ -0,0 +1,11 @@
|
||||
export GRAFANA_URL=""
|
||||
export GRAFANA_TOKEN=""
|
||||
export GRAFANA_DATASOURCE_UID=""
|
||||
export IPFABRIC_URL=""
|
||||
export IPFABRIC_TOKEN=""
|
||||
|
||||
# Optional -- see scripts/README.md for defaults and when to override
|
||||
# export IPFABRIC_SNAPSHOT="$last"
|
||||
# export PROMETHEUS_URL="http://172.16.0.71:9090"
|
||||
# export GRAFANA_DASHBOARD_UID="evpn-vxlan-fabric-weathermap"
|
||||
# export GRAFANA_WEATHERMAP_PLUGIN_ID="tamirsuliman-weathermap-panel"
|
||||
@@ -290,6 +290,25 @@ topology:
|
||||
exec:
|
||||
- ifup eth1
|
||||
|
||||
# =====================================================
|
||||
# TELEMETRY
|
||||
# =====================================================
|
||||
|
||||
gnmic:
|
||||
kind: linux
|
||||
mgmt-ipv4: 172.16.0.70
|
||||
image: ghcr.io/openconfig/gnmic:0.46.0
|
||||
binds:
|
||||
- configs/gnmic/gnmic-config.yml:/gnmic-config.yml:ro
|
||||
cmd: --config /gnmic-config.yml --log subscribe
|
||||
|
||||
prometheus:
|
||||
kind: linux
|
||||
mgmt-ipv4: 172.16.0.71
|
||||
image: prom/prometheus:v2.55.1
|
||||
binds:
|
||||
- configs/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml:ro
|
||||
|
||||
links:
|
||||
# =====================================================
|
||||
# DATACENTER FABRIC LINKS
|
||||
|
||||
167
scripts/README.md
Normal file
167
scripts/README.md
Normal file
@@ -0,0 +1,167 @@
|
||||
# scripts/generate_weathermap.py
|
||||
|
||||
Generates the **weathermap panel only** from live IPFabric + Prometheus
|
||||
data, merges it into a hand-authored dashboard base, and provisions the
|
||||
result to Grafana.
|
||||
|
||||

|
||||
|
||||
## Files
|
||||
|
||||
| File | What it is | Edit it? |
|
||||
| ------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------- |
|
||||
| `configs/grafana/dashboard-base.json` | Hand-authored: BGP table, ports table, throughput graphs, `site`/`device` variables, layout. Has one reserved slot panel (`title: "__WEATHERMAP_SLOT__"`) where the weathermap gets spliced in. | Yes, directly |
|
||||
| `configs/grafana/weathermap-dashboard.json` | Generated build artifact — base + fresh weathermap panel, merged. What actually gets provisioned. | No — regenerate instead |
|
||||
|
||||
Why split like this: topology data (nodes/links) changes with the lab and
|
||||
should regenerate every time; the rest of the dashboard (table layout,
|
||||
thresholds, transformations) is hand-tuned and shouldn't be touched by a
|
||||
topology refresh. See #52.
|
||||
|
||||
## Quickstart
|
||||
|
||||
```bash
|
||||
cp envrc.sample .envrc # fill in the values (see "Credentials" below)
|
||||
source .envrc
|
||||
|
||||
python3 scripts/generate_weathermap.py # dry run: writes the JSON only
|
||||
python3 scripts/generate_weathermap.py --provision # also pushes it to Grafana
|
||||
```
|
||||
|
||||
Re-run the same `--provision` command any time the lab topology changes —
|
||||
the script always rebuilds the weathermap panel from live data, so there's
|
||||
nothing to "detect", just re-run it. It's not automated yet (no watcher/CI
|
||||
hook), so re-running is a manual step for now.
|
||||
|
||||
## Credentials
|
||||
|
||||
| Value | Env var | Get it from the UI | Get it via API |
|
||||
| ----------------------------- | ----------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ |
|
||||
| IPFabric token | `IPFABRIC_TOKEN` | IPFabric → user menu → Settings → API Tokens → Add token | not possible, UI-only |
|
||||
| Grafana service account token | `GRAFANA_TOKEN` | Grafana → Administration → Service accounts → Add → give it **Editor** role → Add token | possible but multi-step (create account, then create token under it) — UI is simpler |
|
||||
| Prometheus datasource UID | `GRAFANA_DATASOURCE_UID` | Grafana → Connections → Data sources → open the one with gnmic data → UID is the last part of the URL | `curl -s -H "Authorization: Bearer $GRAFANA_TOKEN" "$GRAFANA_URL/api/datasources" \| jq '.[] \| {name,uid}'` |
|
||||
| Weathermap plugin ID | `GRAFANA_WEATHERMAP_PLUGIN_ID` (optional, only if not the default fork) | Grafana → Administration → Plugins → search "weathermap" → id is in the URL | — |
|
||||
|
||||
⚠️ The datasource must be the one that can actually see gnmic metrics —
|
||||
picking an unrelated Prometheus instance renders the panel with no data
|
||||
(see #48).
|
||||
|
||||
## Verifying the result
|
||||
|
||||
**In the browser** (the only way to actually see it render — colors,
|
||||
tables, and the ports-table merge transformation all happen client-side):
|
||||
|
||||
1. Open `$GRAFANA_URL/d/evpn-vxlan-fabric-weathermap`.
|
||||
2. Weathermap: nodes grouped spine-top → access-bottom, all green.
|
||||
3. `site`/`device` dropdowns filter the two tables below.
|
||||
4. BGP Sessions / Ports tables: populated, green=Up / red=Down.
|
||||
5. Throughput per Site: signed graph, non-zero values.
|
||||
|
||||
**Without a browser**, use Grafana's **Explore** view (paste a PromQL
|
||||
expression from the panel JSON and run it) — same result as the UI check
|
||||
above minus the rendering, no `curl` needed. Or hit the API directly:
|
||||
|
||||
```bash
|
||||
curl -s -X POST -H "Authorization: Bearer $GRAFANA_TOKEN" -H "Content-Type: application/json" \
|
||||
"$GRAFANA_URL/api/ds/query" -d '{
|
||||
"queries": [{"refId":"A","datasource":{"type":"prometheus","uid":"'"$GRAFANA_DATASOURCE_UID"'"},
|
||||
"expr":"min by (device) (interfaces_interface_state_oper_status)","instant":true}],
|
||||
"from":"now-5m","to":"now"}' | python3 -m json.tool
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Cause | Check |
|
||||
| ---------------------------------------------------------- | ------------------------------------------------------------------- | ------------------------------------------------------------------ |
|
||||
| `Refusing to provision: set GRAFANA_DATASOURCE_UID...` | Env var unset | `echo $GRAFANA_DATASOURCE_UID` |
|
||||
| Panel renders with no data | Wrong datasource — can't see gnmic metrics | Explore → run `up` against it, confirm gnmic series come back |
|
||||
| `Base dashboard has no panel titled '__WEATHERMAP_SLOT__'` | Someone edited `dashboard-base.json` and renamed/removed that panel | Check panel `id: 1` still has that exact title |
|
||||
| Manually-moved node keeps resetting position | `GRAFANA_URL`/`GRAFANA_TOKEN` weren't set for that run | Look for `Fetching live node positions from ...` in the run output |
|
||||
| `HTTP 401/403` fetching live positions | Expired/bad Grafana token | Re-issue the service account token |
|
||||
|
||||
## Environment variables
|
||||
|
||||
| Variable | Required | Default |
|
||||
| ------------------------------ | ------------------ | ------------------------------------- |
|
||||
| `IPFABRIC_URL` | yes | — |
|
||||
| `IPFABRIC_TOKEN` | yes | — |
|
||||
| `IPFABRIC_SNAPSHOT` | no | `$last` |
|
||||
| `PROMETHEUS_URL` | no | `http://172.16.0.71:9090` |
|
||||
| `GRAFANA_URL` | no* | — |
|
||||
| `GRAFANA_TOKEN` | no* | — |
|
||||
| `GRAFANA_DATASOURCE_UID` | with `--provision` | — |
|
||||
| `GRAFANA_DASHBOARD_UID` | no | `evpn-vxlan-fabric-weathermap` |
|
||||
| `GRAFANA_WEATHERMAP_PLUGIN_ID` | no | `tamirsuliman-weathermap-panel` |
|
||||
| `--base` (CLI flag) | no | `configs/grafana/dashboard-base.json` |
|
||||
|
||||
\* `GRAFANA_URL`/`GRAFANA_TOKEN` are required for `--provision`, and
|
||||
optional otherwise — if set on a plain (non-`--provision`) run, they're
|
||||
used to read back live node positions so manual repositioning survives
|
||||
the next regeneration (see #53). Skip them and every node just gets the
|
||||
default layout.
|
||||
|
||||
## How it works
|
||||
|
||||
1. Fetch device inventory + connectivity-matrix from IPFabric.
|
||||
2. Keep only physical Ethernet↔Ethernet links (drop management-plane and
|
||||
`.100`/`.200` subinterface rows), dedupe the two directions IPFabric
|
||||
reports per link.
|
||||
3. Position each node:
|
||||
- **New node** → layered default: Y-tier by role parsed from the
|
||||
hostname (`spine → core → border-leaf → leaf → access`), X grouped
|
||||
by site within the tier. A hostname that matches no role is logged
|
||||
and dropped into a fallback tier, never silently misplaced (#53).
|
||||
- **Known node** → whatever position is currently live in Grafana,
|
||||
unchanged. This is why `GRAFANA_URL`/`GRAFANA_TOKEN` matter even on
|
||||
a dry run (#53).
|
||||
4. Build the panel's PromQL `targets`: node status, per-side link tx (each
|
||||
side its own egress counter, so both directions are represented — #57),
|
||||
VXLAN MAC/VNI tooltip — each with an explicit `legendFormat`.
|
||||
5. Build `nodes[]`/`links[]`, including the plugin's `anchors` tally
|
||||
(link count per side) — required by the installed plugin fork even
|
||||
though the schema doc says it's optional (#48).
|
||||
6. Cross-check every interface name against the live gnmic exporter
|
||||
(`Et`→`Ethernet`, `Po`→`Port-Channel`, etc.); mismatches are logged,
|
||||
not dropped — the link stays, just without data until fixed upstream.
|
||||
7. Load `dashboard-base.json`, substitute the real datasource UID, splice
|
||||
the generated panel into `__WEATHERMAP_SLOT__` (keeping that slot's
|
||||
`gridPos`/`id` — layout stays manual).
|
||||
8. Write `weathermap-dashboard.json`; provision it if `--provision`.
|
||||
|
||||
## Known gaps
|
||||
|
||||
- **VXLAN MAC-per-VNI tooltip**: the `vlan` join key doesn't match on live
|
||||
data for any VTEP node, likely an Arista internal-vs-front-panel VLAN
|
||||
translation gap. Only affects that one decorative tooltip metric, not
|
||||
node/link status or traffic coloring. Tracked in #44.
|
||||
|
||||
# scripts/generate_traffic.sh
|
||||
|
||||
Generates real DC↔Campus traffic over VRF `gold` using `iperf3` (bundled
|
||||
in the `network-multitool` image every host container runs), with a live
|
||||
bandwidth dashboard. Without this, host containers sit idle and IPFabric
|
||||
ARP/MAC tables, Grafana throughput graphs, and the weathermap panel stay
|
||||
empty until someone manually generates traffic (see #55).
|
||||
|
||||
## Quickstart
|
||||
|
||||
```bash
|
||||
./scripts/generate_traffic.sh <duration_seconds>
|
||||
```
|
||||
|
||||
## How it works
|
||||
|
||||
- Starts `iperf3 -s` on the DC gold-VRF servers: `dc-server2`
|
||||
(10.34.34.102), `dc-server4` (10.78.78.104).
|
||||
- Runs `iperf3 -c -R` from the paired campus gold-VRF hosts, reversing the
|
||||
stream so the DC server pushes to the campus consumer: `dc-server2` →
|
||||
`campus-host1`, `dc-server4` → `campus-host2` — exercising the full
|
||||
DC→Core→Campus stitched EVPN Type-5 path end to end.
|
||||
- Redraws a terminal dashboard every second for the run duration: server
|
||||
list, and live Mbits/sec per client session parsed from `iperf3 -i 1`
|
||||
output.
|
||||
- On exit (duration end or Ctrl-C), kills the client processes and stops
|
||||
the `iperf3 -s` processes on the DC servers — no leftover state.
|
||||
|
||||
`dc-server1`/`dc-server3` (VLAN 40, VRF default, no gateway) are out of
|
||||
scope — this script only exercises the routed gold VRF path.
|
||||
88
scripts/generate_traffic.sh
Executable file
88
scripts/generate_traffic.sh
Executable file
@@ -0,0 +1,88 @@
|
||||
#!/usr/bin/env bash
|
||||
# Generates DC<->Campus traffic over VRF gold using iperf3 (bundled in the
|
||||
# network-multitool image every host container runs), with a live
|
||||
# server/client bandwidth dashboard. Refs #55.
|
||||
set -euo pipefail
|
||||
|
||||
DURATION="${1:?Usage: $0 <duration_seconds>}"
|
||||
if ! [[ "$DURATION" =~ ^[0-9]+$ ]] || [[ "$DURATION" -lt 1 ]]; then
|
||||
echo "Duration must be a positive integer (seconds)" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
LAB_PREFIX="clab-arista-evpn-fabric"
|
||||
PORT=5301
|
||||
|
||||
# server_name:server_ip:client_name — gold VRF pairs, stitched EVPN
|
||||
# Type-5 path DC -> Core -> Campus (see README Host Addressing table)
|
||||
PAIRS=(
|
||||
"dc-server2:10.34.34.102:campus-host1"
|
||||
"dc-server4:10.78.78.104:campus-host2"
|
||||
)
|
||||
|
||||
WORKDIR="$(mktemp -d)"
|
||||
CLIENT_PIDS=()
|
||||
|
||||
cleanup() {
|
||||
for pid in "${CLIENT_PIDS[@]:-}"; do
|
||||
kill "$pid" >/dev/null 2>&1 || true
|
||||
done
|
||||
for pair in "${PAIRS[@]}"; do
|
||||
IFS=':' read -r server _ _ <<<"$pair"
|
||||
docker exec "${LAB_PREFIX}-${server}" pkill -f "iperf3 -s -p ${PORT}" >/dev/null 2>&1 || true
|
||||
done
|
||||
rm -rf "$WORKDIR"
|
||||
}
|
||||
trap cleanup EXIT INT TERM
|
||||
|
||||
echo "Starting iperf3 servers..."
|
||||
for pair in "${PAIRS[@]}"; do
|
||||
IFS=':' read -r server server_ip _ <<<"$pair"
|
||||
docker exec -d "${LAB_PREFIX}-${server}" iperf3 -s -p "$PORT"
|
||||
done
|
||||
sleep 1
|
||||
|
||||
echo "Starting iperf3 clients for ${DURATION}s..."
|
||||
for pair in "${PAIRS[@]}"; do
|
||||
IFS=':' read -r server server_ip client <<<"$pair"
|
||||
logfile="${WORKDIR}/${client}.log"
|
||||
# -R: DC hosts the service, campus is the consumer -- data should flow
|
||||
# server -> client (download), not client -> server, to match how a
|
||||
# real DC-hosted service/campus-consumer pair behaves.
|
||||
docker exec "${LAB_PREFIX}-${client}" iperf3 -c "$server_ip" -p "$PORT" -R \
|
||||
-t "$DURATION" -i 1 --forceflush -f m >"$logfile" 2>&1 &
|
||||
CLIENT_PIDS+=("$!")
|
||||
done
|
||||
|
||||
for ((elapsed = 0; elapsed <= DURATION; elapsed++)); do
|
||||
clear
|
||||
echo "EVPN/VXLAN lab traffic generator — ${elapsed}/${DURATION}s"
|
||||
echo
|
||||
echo "Servers (iperf3 -s):"
|
||||
for pair in "${PAIRS[@]}"; do
|
||||
IFS=':' read -r server server_ip _ <<<"$pair"
|
||||
echo " ${server} (${server_ip}:${PORT})"
|
||||
done
|
||||
echo
|
||||
echo "Clients (live bandwidth):"
|
||||
for pair in "${PAIRS[@]}"; do
|
||||
IFS=':' read -r server server_ip client <<<"$pair"
|
||||
logfile="${WORKDIR}/${client}.log"
|
||||
last_line="$(grep -E 'Mbits/sec' "$logfile" 2>/dev/null | tail -1 || true)"
|
||||
bw="$(sed -E 's/.*[[:space:]]([0-9.]+ Mbits\/sec).*/\1/' <<<"$last_line")"
|
||||
[[ -z "$last_line" ]] && bw="waiting..."
|
||||
printf " %-13s -> %-13s : %s\n" "$server" "$client" "$bw"
|
||||
done
|
||||
sleep 1
|
||||
done
|
||||
|
||||
wait "${CLIENT_PIDS[@]}" 2>/dev/null || true
|
||||
|
||||
echo
|
||||
echo "Done. Summary:"
|
||||
for pair in "${PAIRS[@]}"; do
|
||||
IFS=':' read -r server _ client <<<"$pair"
|
||||
logfile="${WORKDIR}/${client}.log"
|
||||
summary="$(grep -E 'receiver' "$logfile" 2>/dev/null || true)"
|
||||
echo " ${server} -> ${client}: ${summary:-no data}"
|
||||
done
|
||||
664
scripts/generate_weathermap.py
Normal file
664
scripts/generate_weathermap.py
Normal file
@@ -0,0 +1,664 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Generate a weathermap-ng PANEL (only) from IPFabric topology + gnmic/Prometheus
|
||||
metrics, merge it into a manually-authored dashboard base, and optionally
|
||||
provision the result into Grafana.
|
||||
|
||||
Scope (see #52): this script knows only about the weathermap panel --
|
||||
targets (node status / link tx per side / VXLAN tooltip queries) and
|
||||
options.weathermap (nodes/links/scale/settings). It has no knowledge of the
|
||||
BGP sessions table, ports/interfaces table, or throughput panels -- those
|
||||
live in the manually-authored `configs/grafana/dashboard-base.json` and are
|
||||
not touched by this script.
|
||||
|
||||
Data sources (see gitea issues #44, #47, #48):
|
||||
- IPFabric REST API: device inventory + connectivity-matrix (topology)
|
||||
- Prometheus (gnmic exporter, configs/prometheus/prometheus.yml): live label
|
||||
values, used both to validate the IPFabric->gnmic interface-name mapping
|
||||
and to resolve which VTEPs/VLANs actually have VXLAN data.
|
||||
|
||||
Merge: the weathermap panel is generated fresh from live data on every run
|
||||
(no diffing), then spliced into the base dashboard's reserved slot (the panel
|
||||
titled "__WEATHERMAP_SLOT__"), keeping that slot's gridPos so the manually
|
||||
authored layout is never repositioned by the generator. See #52.
|
||||
|
||||
Usage:
|
||||
python3 scripts/generate_weathermap.py [--base PATH] [--output PATH] [--provision]
|
||||
|
||||
Environment:
|
||||
IPFABRIC_URL e.g. https://ipfabric.example.com
|
||||
IPFABRIC_TOKEN API token (Inventory/Snapshots read access)
|
||||
IPFABRIC_SNAPSHOT snapshot id, default "$last"
|
||||
PROMETHEUS_URL default http://172.16.0.71:9090 (in-topology instance)
|
||||
GRAFANA_URL required with --provision
|
||||
GRAFANA_TOKEN required with --provision
|
||||
GRAFANA_DATASOURCE_UID Prometheus datasource UID in Grafana, required with --provision
|
||||
GRAFANA_DASHBOARD_UID default "evpn-vxlan-fabric-weathermap"
|
||||
GRAFANA_WEATHERMAP_PLUGIN_ID default "tamirsuliman-weathermap-panel" -- override
|
||||
if a different weathermap-ng fork/plugin id is installed.
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import urllib.request
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
|
||||
WEATHERMAP_SCHEMA_VERSION = 14
|
||||
|
||||
# IPFabric abbreviated interface prefixes -> gnmic/OpenConfig full names.
|
||||
# Longest-prefix-first so e.g. "Ma" doesn't shadow a hypothetical multi-letter clash.
|
||||
INTERFACE_PREFIX_ALIASES = {
|
||||
"Et": "Ethernet",
|
||||
"Po": "Port-Channel",
|
||||
"Lo": "Loopback",
|
||||
"Vl": "Vlan",
|
||||
"Ma": "Management",
|
||||
}
|
||||
|
||||
SITE_ORDER = ["dc", "core", "campus"]
|
||||
GRID_COLUMNS = 6
|
||||
GRID_SPACING_X = 180
|
||||
GRID_SPACING_Y = 150
|
||||
|
||||
# Layered default layout (see #53): device role is parsed from the hostname
|
||||
# naming convention, same trust level as the `site` parsing already in place
|
||||
# for the Prometheus relabel (#49) -- not IPFabric-derived, not configurable.
|
||||
# Checked in this order so e.g. "campus-border-leaf1" matches border-leaf
|
||||
# before the more general leaf pattern.
|
||||
ROLE_PATTERNS = [
|
||||
("spine", re.compile(r"-spine")),
|
||||
("core", re.compile(r"^core")),
|
||||
("border-leaf", re.compile(r"-border-leaf")),
|
||||
("leaf", re.compile(r"-leaf")),
|
||||
("access", re.compile(r"-access")),
|
||||
]
|
||||
ROLE_TIER_ORDER = ["spine", "core", "border-leaf", "leaf", "access"]
|
||||
TIER_SPACING_Y = 300
|
||||
# Per-site X band start, wide enough that the largest role/site group (8 dc
|
||||
# leafs) doesn't spill into the next site's band at GRID_COLUMNS=6.
|
||||
SITE_X_OFFSET = {"dc": 100, "core": 1300, "campus": 1600}
|
||||
UNMATCHED_ROLE_TIER_Y = len(ROLE_TIER_ORDER) * TIER_SPACING_Y + 300
|
||||
|
||||
# tamirsuliman-weathermap-panel's numeric anchor enum (Center=0, Top=1,
|
||||
# Bottom=2, Left=3, Right=4) -- reverse-engineered from module.js, since
|
||||
# link.sides.*.anchor and node.anchors{} both index by this numeric value,
|
||||
# not by the anchor name string.
|
||||
ANCHOR = {"Center": 0, "Top": 1, "Bottom": 2, "Left": 3, "Right": 4}
|
||||
|
||||
NODE_COLORS = {"font": "#ffffff", "background": "#22252b", "border": "#5794F2", "statusDown": "#F2495C"}
|
||||
STATUS_VALUE_MAPPINGS = [{"value": 0, "color": "#F2495C"}, {"value": 1, "color": "#73BF69"}]
|
||||
DEFAULT_LINK_BANDWIDTH_BPS = 10_000_000_000 # 10G fallback when IPFabric speed is missing
|
||||
|
||||
|
||||
def env(name, default=None, required=False):
|
||||
val = os.environ.get(name, default)
|
||||
if required and not val:
|
||||
sys.exit(f"Missing required environment variable: {name}")
|
||||
return val
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# IPFabric
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
def ipfabric_request(base_url, token, path, body):
|
||||
req = urllib.request.Request(
|
||||
f"{base_url.rstrip('/')}/api{path}",
|
||||
data=json.dumps(body).encode(),
|
||||
headers={"X-API-Token": token, "Content-Type": "application/json"},
|
||||
method="POST",
|
||||
)
|
||||
with urllib.request.urlopen(req, timeout=30) as resp:
|
||||
return json.load(resp)["data"]
|
||||
|
||||
|
||||
def fetch_devices(base_url, token, snapshot):
|
||||
return ipfabric_request(
|
||||
base_url, token, "/tables/inventory/devices",
|
||||
{"columns": ["hostname", "siteName", "vendor", "model", "sn"], "snapshot": snapshot,
|
||||
"pagination": {"limit": 1000, "start": 0}},
|
||||
)
|
||||
|
||||
|
||||
def fetch_connectivity_matrix(base_url, token, snapshot):
|
||||
return ipfabric_request(
|
||||
base_url, token, "/tables/interfaces/connectivity-matrix",
|
||||
{"columns": ["localHost", "localInt", "remoteHost", "remoteInt", "protocol"], "snapshot": snapshot,
|
||||
"pagination": {"limit": 5000, "start": 0}},
|
||||
)
|
||||
|
||||
|
||||
def fetch_interface_speeds(base_url, token, snapshot):
|
||||
rows = ipfabric_request(
|
||||
base_url, token, "/tables/inventory/interfaces",
|
||||
{"columns": ["hostname", "intName", "speed"], "snapshot": snapshot,
|
||||
"pagination": {"limit": 5000, "start": 0}},
|
||||
)
|
||||
speeds = {}
|
||||
for row in rows:
|
||||
speed = row.get("speed")
|
||||
if speed:
|
||||
speeds[(row["hostname"], row["intName"])] = int(speed)
|
||||
return speeds
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Prometheus (validation + VXLAN discovery)
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
def prom_query(prometheus_url, promql):
|
||||
url = f"{prometheus_url.rstrip('/')}/api/v1/query?query=" + urllib.parse.quote(promql)
|
||||
with urllib.request.urlopen(url, timeout=15) as resp:
|
||||
payload = json.load(resp)
|
||||
if payload["status"] != "success":
|
||||
sys.exit(f"Prometheus query failed: {promql}")
|
||||
return payload["data"]["result"]
|
||||
|
||||
|
||||
def known_interface_pairs(prometheus_url):
|
||||
"""(device, interface) pairs that actually exist in the gnmic exporter,
|
||||
used to validate the IPFabric->gnmic interface alias before trusting it."""
|
||||
results = prom_query(prometheus_url, "interfaces_interface_state_oper_status")
|
||||
return {(m["metric"]["device"], m["metric"]["interface"]) for m in results}
|
||||
|
||||
|
||||
def known_vtep_vlan_pairs(prometheus_url):
|
||||
"""(device, vlan) pairs with a live VLAN-to-VNI mapping -- these are the
|
||||
VTEP nodes eligible for a VXLAN MAC-count tooltip metric."""
|
||||
results = prom_query(prometheus_url, "interfaces_interface_arista_vxlan_vlan_to_vnis_vlan_to_vni_state_vni")
|
||||
return {(m["metric"]["device"], m["metric"]["vlan"]) for m in results}
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Interface aliasing (IPFabric abbreviated name -> gnmic full name)
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
def alias_interface(ipf_name):
|
||||
match = re.match(r"^([A-Za-z]+)(\d.*)$", ipf_name)
|
||||
if not match:
|
||||
return ipf_name
|
||||
prefix, rest = match.groups()
|
||||
full = INTERFACE_PREFIX_ALIASES.get(prefix)
|
||||
if full is None:
|
||||
return ipf_name
|
||||
return f"{full}{rest}"
|
||||
|
||||
|
||||
def promql_escape(s):
|
||||
"""re.escape() escapes '-' as '\\-', which Python's own regex engine
|
||||
accepts but PromQL's RE2-based engine rejects ("unknown escape sequence
|
||||
U+002D '-'") -- and every hostname in this lab contains hyphens. '-' is
|
||||
not a regex metacharacter outside a character class, so it's safe to
|
||||
leave unescaped."""
|
||||
return re.escape(s).replace("\\-", "-")
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Topology processing
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
def parse_role(hostname):
|
||||
"""Device role from the hostname naming convention -- see ROLE_PATTERNS.
|
||||
Returns None if nothing matches (logged by the caller, not silently
|
||||
defaulted into the wrong tier)."""
|
||||
for role, pattern in ROLE_PATTERNS:
|
||||
if pattern.search(hostname):
|
||||
return role
|
||||
return None
|
||||
|
||||
|
||||
def build_layout(devices, mismatches):
|
||||
"""Default layered layout for nodes with no live (Grafana-drag-and-drop)
|
||||
position yet -- see #53. Y-tier by role (spine top, access bottom), X
|
||||
grouped/columned by site within each tier so same-role devices from
|
||||
different sites don't overlap."""
|
||||
groups = {} # (role, site) -> [hostname, ...]
|
||||
unmatched = []
|
||||
for dev in devices:
|
||||
host, site = dev["hostname"], dev["siteName"]
|
||||
role = parse_role(host)
|
||||
if role is None:
|
||||
unmatched.append(host)
|
||||
continue
|
||||
groups.setdefault((role, site), []).append(host)
|
||||
|
||||
positions = {}
|
||||
for role_idx, role in enumerate(ROLE_TIER_ORDER):
|
||||
tier_y = role_idx * TIER_SPACING_Y
|
||||
for site in SITE_ORDER:
|
||||
x_offset = SITE_X_OFFSET.get(site, max(SITE_X_OFFSET.values()) + 300)
|
||||
for idx, host in enumerate(sorted(groups.get((role, site), []))):
|
||||
col, row = idx % GRID_COLUMNS, idx // GRID_COLUMNS
|
||||
positions[host] = [x_offset + col * GRID_SPACING_X, tier_y + row * GRID_SPACING_Y]
|
||||
|
||||
if unmatched:
|
||||
mismatches.append(
|
||||
f"{len(unmatched)} hostname(s) matched no role pattern (spine/core/border-leaf/leaf/access), "
|
||||
f"placed in a fallback tier instead of guessing: {', '.join(sorted(unmatched))}"
|
||||
)
|
||||
for idx, host in enumerate(sorted(unmatched)):
|
||||
col, row = idx % GRID_COLUMNS, idx // GRID_COLUMNS
|
||||
positions[host] = [100 + col * GRID_SPACING_X, UNMATCHED_ROLE_TIER_Y + row * GRID_SPACING_Y]
|
||||
|
||||
return positions
|
||||
|
||||
|
||||
def dedupe_links(connectivity_matrix):
|
||||
"""connectivity-matrix reports each physical link twice (once from each
|
||||
side), plus management-plane (Management0) neighbor entries, plus one row
|
||||
per 802.1Q subinterface on trunked ports (e.g. Et13.100/Et13.200, used for
|
||||
the gold VRF stitching on Core -- see README). Subinterfaces ride the same
|
||||
physical port as their parent interface, so they'd otherwise show up as
|
||||
bogus duplicate parallel links with permanently-unresolvable queries
|
||||
(gnmic only subscribes to physical interface counters). Keep only
|
||||
physical Ethernet-to-Ethernet links, one entry per unordered
|
||||
(host,int)-(host,int) pair."""
|
||||
seen = set()
|
||||
links = []
|
||||
for row in connectivity_matrix:
|
||||
local_int, remote_int = row["localInt"], row["remoteInt"]
|
||||
if "." in local_int or "." in remote_int:
|
||||
continue
|
||||
if not (local_int.startswith("Et") and remote_int.startswith("Et")):
|
||||
continue
|
||||
side_a = (row["localHost"], local_int)
|
||||
side_z = (row["remoteHost"], remote_int)
|
||||
key = tuple(sorted([side_a, side_z]))
|
||||
if key in seen:
|
||||
continue
|
||||
seen.add(key)
|
||||
links.append({"a_host": key[0][0], "a_int": key[0][1], "z_host": key[1][0], "z_int": key[1][1]})
|
||||
return links
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Weathermap assembly
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
def build_weathermap(devices, links, interface_speeds, positions, prometheus_url, mismatches):
|
||||
hostnames = [d["hostname"] for d in devices]
|
||||
host_regex = "|".join(promql_escape(h) for h in hostnames)
|
||||
|
||||
known_pairs = known_interface_pairs(prometheus_url)
|
||||
vtep_vlan_pairs = known_vtep_vlan_pairs(prometheus_url)
|
||||
vtep_hosts = sorted({dev for dev, _ in vtep_vlan_pairs})
|
||||
|
||||
def resolve_and_check(host, ipf_int, side_label):
|
||||
gnmic_int = alias_interface(ipf_int)
|
||||
if (host, gnmic_int) not in known_pairs:
|
||||
mismatches.append(
|
||||
f"{host} {ipf_int} -> {gnmic_int} ({side_label}): no matching series in "
|
||||
f"interfaces_interface_state_oper_status -- link will reference a query that "
|
||||
f"resolves to no data until the gnmic/IPFabric interface names are aligned "
|
||||
f"(see #47/#48 interface aliasing fallback)"
|
||||
)
|
||||
return gnmic_int
|
||||
|
||||
# -- nodes --
|
||||
# anchors{} tallies how many link-sides attach to each anchor position on
|
||||
# this node; every link below uses Right for its A side and Left for its
|
||||
# Z side, so that's what gets counted per node here.
|
||||
anchor_counts = {host: {a: 0 for a in ANCHOR.values()} for host in (d["hostname"] for d in devices)}
|
||||
for link in links:
|
||||
anchor_counts[link["a_host"]][ANCHOR["Right"]] += 1
|
||||
anchor_counts[link["z_host"]][ANCHOR["Left"]] += 1
|
||||
|
||||
nodes = []
|
||||
for dev in devices:
|
||||
host = dev["hostname"]
|
||||
node = {
|
||||
"id": host,
|
||||
"label": host,
|
||||
"position": positions[host],
|
||||
"isConnection": False,
|
||||
"useConstantSpacing": False,
|
||||
# True (not the plugin default) so rendered node HEIGHT is
|
||||
# constant, decoupled from per-node link count -- see #54. Ground
|
||||
# truth confirmed against the installed plugin's real module.js
|
||||
# (not the #48 schema doc, which doesn't cover this): height is
|
||||
# `fontSize + 2*padding.vertical` when compactVerticalLinks is
|
||||
# true, unconditionally; when false, it's the larger of that or
|
||||
# a term proportional to max(anchors[Left].numLinks,
|
||||
# anchors[Right].numLinks) -- exactly why campus-leaf1 (3 links)
|
||||
# and campus-leaf2 (4 links) rendered at different heights.
|
||||
# Width is unaffected either way: it's always recomputed from
|
||||
# the label text at render time (no override field exists), and
|
||||
# useConstantSpacing only pulls in Top/Bottom anchor link count,
|
||||
# which this generator never uses (links always attach
|
||||
# Right/Left -- see anchor_counts above).
|
||||
"compactVerticalLinks": True,
|
||||
"padding": {"horizontal": 12, "vertical": 6},
|
||||
"colors": dict(NODE_COLORS),
|
||||
"nodeIcon": None,
|
||||
"statusQuery": f"STATUS {host}",
|
||||
"nodeStatusColorTarget": "border",
|
||||
"statusValueMappings": [dict(m) for m in STATUS_VALUE_MAPPINGS],
|
||||
"anchors": {a: {"numLinks": anchor_counts[host][a], "numFilledLinks": 0} for a in ANCHOR.values()},
|
||||
}
|
||||
if host in vtep_hosts:
|
||||
node["tooltipMetrics"] = [
|
||||
{"label": f"VNI {host} {vlan}", "query": f"VNI {host} {vlan}", "units": "MACs"}
|
||||
for _, vlan in sorted(v for v in vtep_vlan_pairs if v[0] == host)
|
||||
]
|
||||
nodes.append(node)
|
||||
|
||||
# -- links --
|
||||
link_defs = []
|
||||
interface_regex_parts = set()
|
||||
for link in links:
|
||||
a_gnmic_int = resolve_and_check(link["a_host"], link["a_int"], "side A")
|
||||
z_gnmic_int = resolve_and_check(link["z_host"], link["z_int"], "side Z")
|
||||
interface_regex_parts.add(promql_escape(a_gnmic_int))
|
||||
interface_regex_parts.add(promql_escape(z_gnmic_int))
|
||||
|
||||
a_bw = interface_speeds.get((link["a_host"], link["a_int"]), DEFAULT_LINK_BANDWIDTH_BPS)
|
||||
z_bw = interface_speeds.get((link["z_host"], link["z_int"]), DEFAULT_LINK_BANDWIDTH_BPS)
|
||||
if (link["a_host"], link["a_int"]) not in interface_speeds:
|
||||
mismatches.append(f"{link['a_host']} {link['a_int']}: no IPFabric speed, defaulting bandwidth to {DEFAULT_LINK_BANDWIDTH_BPS}")
|
||||
if (link["z_host"], link["z_int"]) not in interface_speeds:
|
||||
mismatches.append(f"{link['z_host']} {link['z_int']}: no IPFabric speed, defaulting bandwidth to {DEFAULT_LINK_BANDWIDTH_BPS}")
|
||||
|
||||
link_defs.append({
|
||||
"id": f"{link['a_host']}-{a_gnmic_int}--{link['z_host']}-{z_gnmic_int}",
|
||||
"nodes": [{"id": link["a_host"]}, {"id": link["z_host"]}],
|
||||
# Each side's query is that side's own tx (egress) counter, not the
|
||||
# far end's rx -- see #57. a_host tx and z_host rx both describe the
|
||||
# *same* A->Z flow measured from opposite ends (the a_host->z_host
|
||||
# direction, counted twice), leaving the Z->A direction never
|
||||
# queried by either side. Using each node's own tx gives two
|
||||
# independent, opposite-direction measurements instead.
|
||||
"sides": {
|
||||
"A": {
|
||||
"bandwidth": a_bw,
|
||||
"query": f"{link['a_host']} {a_gnmic_int} tx",
|
||||
"labelOffset": 55, "anchor": ANCHOR["Right"], "dashboardLink": "",
|
||||
},
|
||||
"Z": {
|
||||
"bandwidth": z_bw,
|
||||
"query": f"{link['z_host']} {z_gnmic_int} tx",
|
||||
"labelOffset": 55, "anchor": ANCHOR["Left"], "dashboardLink": "",
|
||||
},
|
||||
},
|
||||
"units": "bps",
|
||||
"arrows": {"width": 8, "height": 10, "offset": 2},
|
||||
"stroke": 5,
|
||||
"showThroughputPercentage": False,
|
||||
})
|
||||
|
||||
interface_regex = "|".join(sorted(interface_regex_parts))
|
||||
|
||||
# -- targets (one query per metric family, per task spec) --
|
||||
#
|
||||
# Node status (refId A): a device is "up" only if at least one interface
|
||||
# is up AND every BGP session is up, if it has any -- see #49. The old
|
||||
# "BGP {{device}}" query sourced raw per-neighbor session-state series,
|
||||
# so a device with multiple neighbors collided on one legend string and
|
||||
# only one arbitrarily won. `min by (device)` picks the *worst* session
|
||||
# (0 beats 1), and devices with zero BGP sessions (access switches) are
|
||||
# not penalized: the `or` fallback substitutes a constant 1 for any
|
||||
# device present in device-up but absent from the BGP series entirely.
|
||||
device_up = f'min by (device) (interfaces_interface_state_oper_status{{device=~"{host_regex}"}})'
|
||||
bgp_worst = (
|
||||
f'min by (device) (network_instances_network_instance_protocols_protocol_bgp_neighbors_neighbor_state_session_state'
|
||||
f'{{network_instance_name="default", device=~"{host_regex}"}})'
|
||||
)
|
||||
bgp_ok_or_not_applicable = f'({bgp_worst} or ({device_up} * 0 + 1))'
|
||||
targets = [
|
||||
{
|
||||
"refId": "A",
|
||||
"expr": f'{device_up} * {bgp_ok_or_not_applicable}',
|
||||
"legendFormat": "STATUS {{device}}",
|
||||
},
|
||||
{
|
||||
"refId": "B",
|
||||
"expr": f'rate(interfaces_interface_state_counters_out_octets{{device=~"{host_regex}", interface=~"{interface_regex}"}}[5m]) * 8',
|
||||
"legendFormat": "{{device}} {{interface}} tx",
|
||||
},
|
||||
]
|
||||
if vtep_hosts:
|
||||
vtep_regex = "|".join(promql_escape(h) for h in vtep_hosts)
|
||||
# Verbatim join from #44 "VXLAN (MAC count per VNI)", scoped to VTEP nodes.
|
||||
# RHS is wrapped in max by (device, vlan) so the join key is always
|
||||
# unique -- a Prometheus restart or relabel change otherwise leaves
|
||||
# the pre-restart (frozen) and post-restart series briefly coexisting
|
||||
# within the 5m staleness window, both matching the same (device,
|
||||
# vlan) group, which trips PromQL's "many-to-many matching not
|
||||
# allowed" error. See #50.
|
||||
targets.append({
|
||||
"refId": "D",
|
||||
"expr": (
|
||||
f'count by (device, vlan) (network_instances_network_instance_fdb_mac_table_entries_entry_vlan{{device=~"{vtep_regex}"}})\n'
|
||||
f'* on(device, vlan) group_left(vlan_to_vni_state_vni)\n'
|
||||
f'max by (device, vlan) (interfaces_interface_arista_vxlan_vlan_to_vnis_vlan_to_vni_state_vni{{device=~"{vtep_regex}"}})'
|
||||
),
|
||||
"legendFormat": "VNI {{device}} {{vlan}}",
|
||||
})
|
||||
|
||||
weathermap = {
|
||||
"version": WEATHERMAP_SCHEMA_VERSION,
|
||||
"id": "evpn-vxlan-fabric-weathermap",
|
||||
"nodes": nodes,
|
||||
"links": link_defs,
|
||||
"scale": [
|
||||
{"percent": 0, "color": "#5794F2"},
|
||||
{"percent": 70, "color": "#FA6400"},
|
||||
{"percent": 90, "color": "#C4162A"},
|
||||
],
|
||||
"settings": {
|
||||
"panel": {"backgroundColor": "#212124", "panelSize": {"width": 1600, "height": 1000},
|
||||
"zoomScale": 0, "offset": {"x": 0, "y": 0}, "showTimestamp": True,
|
||||
"grid": {"enabled": False, "size": 10, "guidesEnabled": False}},
|
||||
"link": {"spacing": {"horizontal": 10, "vertical": 5}, "stroke": {"color": "#CCCCDC"},
|
||||
"label": {"background": "#FFFFFF", "border": "#000000", "font": "#000000"},
|
||||
"showAllWithPercentage": False, "defaultUnits": "bps"},
|
||||
"tooltip": {"fontSize": 10, "textColor": "#CCCCDC", "backgroundColor": "#1A1B1F",
|
||||
"inboundColor": "#73BF69", "outboundColor": "#5794F2", "scaleToBandwidth": False},
|
||||
"fontSizing": {"node": 12, "link": 10},
|
||||
"scale": {"position": {"x": 0, "y": 0}, "size": {"width": 150, "height": 100},
|
||||
"title": "Utilization", "fontSizing": {"title": 10, "threshold": 9}},
|
||||
},
|
||||
}
|
||||
return weathermap, targets
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Weathermap panel (only) -- see #52, scope narrowed from full-dashboard
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
WEATHERMAP_SLOT_TITLE = "__WEATHERMAP_SLOT__"
|
||||
WEATHERMAP_PANEL_TITLE = "Fabric Weathermap" # what the slot is renamed to on merge
|
||||
|
||||
|
||||
def build_weathermap_panel(weathermap, targets, datasource_uid, plugin_id):
|
||||
"""The weathermap panel's own content: type/datasource/targets/options.
|
||||
Deliberately has no gridPos/id/title of its own -- those come from
|
||||
whatever slot it's merged into (see merge_weathermap_into_base)."""
|
||||
return {
|
||||
"type": plugin_id,
|
||||
"datasource": {"type": "prometheus", "uid": datasource_uid},
|
||||
"targets": [dict(t, datasource={"type": "prometheus", "uid": datasource_uid}) for t in targets],
|
||||
"options": {"weathermap": weathermap},
|
||||
}
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Manual base dashboard + merge (see #52)
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
def load_base_dashboard(path):
|
||||
with open(path) as f:
|
||||
return json.load(f)
|
||||
|
||||
|
||||
def substitute_placeholders(obj, replacements):
|
||||
"""Recursively replace exact-match string placeholders (e.g. the
|
||||
datasource UID token) anywhere in the manually-authored base JSON."""
|
||||
if isinstance(obj, dict):
|
||||
return {k: substitute_placeholders(v, replacements) for k, v in obj.items()}
|
||||
if isinstance(obj, list):
|
||||
return [substitute_placeholders(v, replacements) for v in obj]
|
||||
if isinstance(obj, str) and obj in replacements:
|
||||
return replacements[obj]
|
||||
return obj
|
||||
|
||||
|
||||
def merge_weathermap_into_base(base_dashboard, weathermap_panel, dashboard_uid):
|
||||
"""Splice the freshly generated weathermap panel into the base
|
||||
dashboard's reserved slot (matched by title), keeping the slot's
|
||||
gridPos/id -- layout stays manual, the generator never repositions it."""
|
||||
dashboard = dict(base_dashboard)
|
||||
dashboard["uid"] = dashboard_uid
|
||||
panels = []
|
||||
found = False
|
||||
for panel in dashboard.get("panels", []):
|
||||
if panel.get("title") == WEATHERMAP_SLOT_TITLE:
|
||||
found = True
|
||||
merged = dict(panel)
|
||||
merged.update(weathermap_panel)
|
||||
merged["title"] = WEATHERMAP_PANEL_TITLE
|
||||
panels.append(merged)
|
||||
else:
|
||||
panels.append(panel)
|
||||
if not found:
|
||||
sys.exit(f"Base dashboard has no panel titled {WEATHERMAP_SLOT_TITLE!r} -- nowhere to merge the weathermap panel")
|
||||
dashboard["panels"] = panels
|
||||
return dashboard
|
||||
|
||||
|
||||
def fetch_live_node_positions(grafana_url, api_token, dashboard_uid):
|
||||
"""Node positions as currently provisioned in Grafana, keyed by node id
|
||||
-- see #53. Position is edited live via drag-and-drop in the Grafana UI,
|
||||
not in the git-committed base file, so it's the one weathermap field that
|
||||
must be sourced from live Grafana state rather than regenerated or read
|
||||
from the manual base -- a manual repositioning must survive every rerun.
|
||||
|
||||
Returns {} if the dashboard doesn't exist yet (first-ever run) or has no
|
||||
weathermap panel yet -- everything falls back to the default layered
|
||||
layout in that case. Any other HTTP error is treated as a real
|
||||
misconfiguration (e.g. a bad token) and raised, rather than silently
|
||||
treated as "no dashboard" -- that would risk quietly discarding every
|
||||
manual position on a run that should have failed loudly instead."""
|
||||
req = urllib.request.Request(
|
||||
f"{grafana_url.rstrip('/')}/api/dashboards/uid/{dashboard_uid}",
|
||||
headers={"Authorization": f"Bearer {api_token}"},
|
||||
method="GET",
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=30) as resp:
|
||||
payload = json.load(resp)
|
||||
except urllib.error.HTTPError as e:
|
||||
if e.code == 404:
|
||||
return {}
|
||||
sys.exit(f"Fetching live dashboard for position preservation failed: HTTP {e.code} {e.read().decode()}")
|
||||
|
||||
for panel in payload.get("dashboard", {}).get("panels", []):
|
||||
if panel.get("title") == WEATHERMAP_PANEL_TITLE:
|
||||
nodes = panel.get("options", {}).get("weathermap", {}).get("nodes", [])
|
||||
return {n["id"]: n["position"] for n in nodes if "position" in n}
|
||||
return {}
|
||||
|
||||
|
||||
def provision_to_grafana(grafana_url, api_token, dashboard_payload):
|
||||
req = urllib.request.Request(
|
||||
f"{grafana_url.rstrip('/')}/api/dashboards/db",
|
||||
data=json.dumps(dashboard_payload).encode(),
|
||||
headers={"Authorization": f"Bearer {api_token}", "Content-Type": "application/json"},
|
||||
method="POST",
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=30) as resp:
|
||||
return json.load(resp)
|
||||
except urllib.error.HTTPError as e:
|
||||
sys.exit(f"Grafana provisioning failed: HTTP {e.code} {e.read().decode()}")
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
parser.add_argument("--base", default="configs/grafana/dashboard-base.json",
|
||||
help="manually-authored dashboard base JSON (everything but the weathermap panel)")
|
||||
parser.add_argument("--output", default="configs/grafana/weathermap-dashboard.json",
|
||||
help="where to write the merged dashboard-as-code JSON (build artifact)")
|
||||
parser.add_argument("--provision", action="store_true",
|
||||
help="also POST the merged dashboard to the Grafana API (requires GRAFANA_* env vars)")
|
||||
args = parser.parse_args()
|
||||
|
||||
ipfabric_url = env("IPFABRIC_URL", required=True)
|
||||
ipfabric_token = env("IPFABRIC_TOKEN", required=True)
|
||||
snapshot = env("IPFABRIC_SNAPSHOT", "$last")
|
||||
prometheus_url = env("PROMETHEUS_URL", "http://172.16.0.71:9090")
|
||||
dashboard_uid = env("GRAFANA_DASHBOARD_UID", "evpn-vxlan-fabric-weathermap")
|
||||
datasource_uid = env("GRAFANA_DATASOURCE_UID", "PROMETHEUS_DATASOURCE_UID_PLACEHOLDER")
|
||||
plugin_id = env("GRAFANA_WEATHERMAP_PLUGIN_ID", "tamirsuliman-weathermap-panel")
|
||||
grafana_url = env("GRAFANA_URL")
|
||||
grafana_token = env("GRAFANA_TOKEN")
|
||||
|
||||
print(f"Fetching devices from IPFabric ({ipfabric_url}, snapshot={snapshot})...")
|
||||
devices = fetch_devices(ipfabric_url, ipfabric_token, snapshot)
|
||||
print(f" {len(devices)} devices")
|
||||
|
||||
print("Fetching connectivity-matrix...")
|
||||
matrix = fetch_connectivity_matrix(ipfabric_url, ipfabric_token, snapshot)
|
||||
links = dedupe_links(matrix)
|
||||
print(f" {len(links)} fabric links after Management-plane filter + dedup ({len(matrix)} raw rows)")
|
||||
|
||||
print("Fetching interface speeds...")
|
||||
interface_speeds = fetch_interface_speeds(ipfabric_url, ipfabric_token, snapshot)
|
||||
|
||||
mismatches = []
|
||||
default_positions = build_layout(devices, mismatches)
|
||||
|
||||
# Position specifically is edited live in the Grafana UI (drag-and-drop),
|
||||
# not in the git-committed base file -- see #53. Reuse whatever's live
|
||||
# for nodes that already exist there; only brand-new nodes get the
|
||||
# layered default. Iterating over default_positions (this run's device
|
||||
# set) rather than the live map means a removed device's stale live
|
||||
# position is simply never looked up again, no orphaned entry survives.
|
||||
if grafana_url and grafana_token:
|
||||
print(f"Fetching live node positions from {grafana_url} (preserve manual repositioning)...")
|
||||
live_positions = fetch_live_node_positions(grafana_url, grafana_token, dashboard_uid)
|
||||
print(f" {len(live_positions)} node(s) with an existing live position")
|
||||
else:
|
||||
print("GRAFANA_URL/GRAFANA_TOKEN not set -- skipping live position fetch, using layered default for all nodes", file=sys.stderr)
|
||||
live_positions = {}
|
||||
positions = {host: live_positions.get(host, default) for host, default in default_positions.items()}
|
||||
|
||||
print(f"Cross-checking interface names against live exporter ({prometheus_url})...")
|
||||
weathermap, targets = build_weathermap(devices, links, interface_speeds, positions, prometheus_url, mismatches)
|
||||
|
||||
if mismatches:
|
||||
print(f"\n{len(mismatches)} mismatch(es) found (link/position kept, not dropped):", file=sys.stderr)
|
||||
for m in mismatches:
|
||||
print(f" - {m}", file=sys.stderr)
|
||||
print(file=sys.stderr)
|
||||
|
||||
weathermap_panel = build_weathermap_panel(weathermap, targets, datasource_uid, plugin_id)
|
||||
|
||||
print(f"Loading manual dashboard base ({args.base})...")
|
||||
base_dashboard = load_base_dashboard(args.base)
|
||||
base_dashboard = substitute_placeholders(base_dashboard, {"__DATASOURCE_UID__": datasource_uid})
|
||||
dashboard = merge_weathermap_into_base(base_dashboard, weathermap_panel, dashboard_uid)
|
||||
dashboard_payload = {"dashboard": dashboard, "overwrite": True}
|
||||
|
||||
os.makedirs(os.path.dirname(args.output) or ".", exist_ok=True)
|
||||
with open(args.output, "w") as f:
|
||||
json.dump(dashboard_payload, f, indent=2)
|
||||
f.write("\n")
|
||||
print(f"Wrote {args.output} ({len(weathermap['nodes'])} nodes, {len(weathermap['links'])} links, "
|
||||
f"{len(dashboard['panels'])} panels total)")
|
||||
|
||||
if args.provision:
|
||||
if not (grafana_url and grafana_token):
|
||||
sys.exit("Refusing to provision: GRAFANA_URL and GRAFANA_TOKEN must both be set")
|
||||
if datasource_uid == "PROMETHEUS_DATASOURCE_UID_PLACEHOLDER":
|
||||
sys.exit("Refusing to provision: set GRAFANA_DATASOURCE_UID to the real Prometheus datasource UID first")
|
||||
print(f"Provisioning dashboard '{dashboard_uid}' to {grafana_url}...")
|
||||
result = provision_to_grafana(grafana_url, grafana_token, dashboard_payload)
|
||||
print(f" {result.get('status')}: {result.get('url')}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in New Issue
Block a user