diff --git a/README.md b/README.md index 00aaaf1..60695d9 100644 --- a/README.md +++ b/README.md @@ -1,10 +1,14 @@ -# Complete IP Address Plan - Arista L5 Dual DC +# Complete IP Address Plan - Arista L5 Dual DC with Access Layer ## 🎯 Design Philosophy -**Avoiding 192.168.x.x to prevent conflicts with LAN subnet (192.168.1.0/24)** +**4-Tier Data Center Architecture**: +- **Spine Layer**: Core routing (L3) +- **Leaf Layer**: Aggregation/Distribution (L3 + VXLAN VTEPs) +- **Access Layer**: Rack/Bay switches (L2/L3 - per bay/rack) +- **Host Layer**: End servers/workloads -We'll use **RFC 1918 private address space** strategically: +**IP Strategy**: - **Management**: `10.255.0.0/24` (out-of-band) - **DC1 Underlay**: `10.1.x.x/16` range - **DC2 Underlay**: `10.2.x.x/16` range @@ -15,20 +19,22 @@ We'll use **RFC 1918 private address space** strategically: ## 📊 IP Address Summary Table -| Network Purpose | Subnet | Size | Usage | -|----------------|--------|------|-------| -| Management (OOB) | 10.255.0.0/24 | 254 hosts | ContainerLab mgmt | -| DC1 Loopback0 | 10.1.0.0/24 | 254 hosts | Router IDs | -| DC1 Loopback1 | 10.1.1.0/24 | 254 hosts | VTEP addresses | -| DC1 P2P Links | 10.1.10.0/24 | 127 /31s | Spine-Leaf links | -| DC1 MLAG Peer | 10.1.255.0/30 | Per pair | MLAG peer links | -| DC2 Loopback0 | 10.2.0.0/24 | 254 hosts | Router IDs | -| DC2 Loopback1 | 10.2.1.0/24 | 254 hosts | VTEP addresses | -| DC2 P2P Links | 10.2.10.0/24 | 127 /31s | Spine-Leaf links | -| DC2 MLAG Peer | 10.2.255.0/30 | Per pair | MLAG peer links | -| DCI Loopback | 10.253.0.1/32 | 1 host | DCI router ID | -| DCI P2P Links | 10.253.254.0/24 | 127 /31s | Border-DCI links | -| Tenant VLANs | 172.16.x.0/24 | Per VLAN | Host networks | +| Network Purpose | Subnet | Size | Usage | +| ------------------- | --------------- | --------- | ---------------------------- | +| Management (OOB) | 10.255.0.0/24 | 254 hosts | ContainerLab mgmt | +| DC1 Loopback0 | 10.1.0.0/24 | 254 hosts | Router IDs (Spine+Leaf only) | +| DC1 Loopback1 | 10.1.1.0/24 | 254 hosts | VTEP addresses (Leaf only) | +| DC1 Spine-Leaf P2P | 10.1.10.0/24 | 127 /31s | Underlay links | +| DC1 Leaf-Access P2P | 10.1.20.0/24 | 127 /31s | Access uplinks | +| DC1 MLAG Peer | 10.1.255.0/30 | Per pair | MLAG peer links | +| DC2 Loopback0 | 10.2.0.0/24 | 254 hosts | Router IDs (Spine+Leaf only) | +| DC2 Loopback1 | 10.2.1.0/24 | 254 hosts | VTEP addresses (Leaf only) | +| DC2 Spine-Leaf P2P | 10.2.10.0/24 | 127 /31s | Underlay links | +| DC2 Leaf-Access P2P | 10.2.20.0/24 | 127 /31s | Access uplinks | +| DC2 MLAG Peer | 10.2.255.0/30 | Per pair | MLAG peer links | +| DCI Loopback | 10.253.0.1/32 | 1 host | DCI router ID | +| DCI P2P Links | 10.253.254.0/24 | 127 /31s | Border-DCI links | +| Tenant VLANs | 172.16.x.0/24 | Per VLAN | Host networks | --- @@ -38,177 +44,201 @@ We'll use **RFC 1918 private address space** strategically: ### DC1 Management IPs ``` -Device | Management IP | Purpose ---------------------|----------------|------------------ -spine1-DC1 | 10.255.0.11 | SSH/API access -spine2-DC1 | 10.255.0.12 | SSH/API access -spine3-DC1 | 10.255.0.13 | SSH/API access -leaf1-DC1 | 10.255.0.21 | SSH/API access -leaf2-DC1 | 10.255.0.22 | SSH/API access -leaf3-DC1 | 10.255.0.23 | SSH/API access -leaf4-DC1 | 10.255.0.24 | SSH/API access -borderleaf1-DC1 | 10.255.0.31 | SSH/API access -borderleaf2-DC1 | 10.255.0.32 | SSH/API access +| Device | Management IP | Layer | Purpose | +| --------------- | ------------- | ----------- | -------------- | +| spine1-DC1 | 10.255.0.11 | Spine | SSH/API access | +| spine2-DC1 | 10.255.0.12 | Spine | SSH/API access | +| spine3-DC1 | 10.255.0.13 | Spine | SSH/API access | +| leaf1-DC1 | 10.255.0.21 | Leaf/Agg | SSH/API access | +| leaf2-DC1 | 10.255.0.22 | Leaf/Agg | SSH/API access | +| leaf3-DC1 | 10.255.0.23 | Leaf/Agg | SSH/API access | +| leaf4-DC1 | 10.255.0.24 | Leaf/Agg | SSH/API access | +| borderleaf1-DC1 | 10.255.0.31 | Border/DCI | SSH/API access | +| borderleaf2-DC1 | 10.255.0.32 | Border/DCI | SSH/API access | +| access1-DC1 | 10.255.0.71 | Access/Rack | SSH/API access | +| access2-DC1 | 10.255.0.72 | Access/Rack | SSH/API access | ``` ### DC2 Management IPs ``` -Device | Management IP | Purpose ---------------------|----------------|------------------ -spine1-DC2 | 10.255.0.41 | SSH/API access -spine2-DC2 | 10.255.0.42 | SSH/API access -spine3-DC2 | 10.255.0.43 | SSH/API access -leaf1-DC2 | 10.255.0.51 | SSH/API access -leaf2-DC2 | 10.255.0.52 | SSH/API access -leaf3-DC2 | 10.255.0.53 | SSH/API access -leaf4-DC2 | 10.255.0.54 | SSH/API access -borderleaf1-DC2 | 10.255.0.61 | SSH/API access -borderleaf2-DC2 | 10.255.0.62 | SSH/API access +| Device | Management IP | Layer | Purpose | +| --------------- | ------------- | ----------- | -------------- | +| spine1-DC2 | 10.255.0.41 | Spine | SSH/API access | +| spine2-DC2 | 10.255.0.42 | Spine | SSH/API access | +| spine3-DC2 | 10.255.0.43 | Spine | SSH/API access | +| leaf1-DC2 | 10.255.0.51 | Leaf/Agg | SSH/API access | +| leaf2-DC2 | 10.255.0.52 | Leaf/Agg | SSH/API access | +| leaf3-DC2 | 10.255.0.53 | Leaf/Agg | SSH/API access | +| leaf4-DC2 | 10.255.0.54 | Leaf/Agg | SSH/API access | +| borderleaf1-DC2 | 10.255.0.61 | Border/DCI | SSH/API access | +| borderleaf2-DC2 | 10.255.0.62 | Border/DCI | SSH/API access | +| access1-DC2 | 10.255.0.81 | Access/Rack | SSH/API access | +| access2-DC2 | 10.255.0.82 | Access/Rack | SSH/API access | ``` ### DCI Management IP ``` -Device | Management IP | Purpose ---------------------|----------------|------------------ -DCI | 10.255.0.100 | SSH/API access +| Device | Management IP | Purpose | +| ------ | ------------- | -------------- | +| DCI | 10.255.0.100 | SSH/API access | ``` ### Host Management IPs ``` -Device | Management IP | Purpose ---------------------|----------------|------------------ -host1-DC1 | 10.255.0.201 | SSH access -host2-DC1 | 10.255.0.202 | SSH access -host1-DC2 | 10.255.0.211 | SSH access -host2-DC2 | 10.255.0.212 | SSH access +| Device | Management IP | Rack/Bay | Purpose | +| --------- | ------------- | -------- | ---------- | +| host1-DC1 | 10.255.0.201 | Bay 1 | SSH access | +| host2-DC1 | 10.255.0.202 | Bay 2 | SSH access | +| host1-DC2 | 10.255.0.211 | Bay 1 | SSH access | +| host2-DC2 | 10.255.0.212 | Bay 2 | SSH access | ``` --- ## 🏢 DC1 - Data Center 1 IP Plan -### Loopback0 Addresses (Router IDs) +### Loopback0 Addresses (Router IDs - Spine & Leaf Only) **Subnet**: `10.1.0.0/24` ``` -Device | Loopback0 | Mask | Router ID ---------------------|----------------|------|---------- -spine1-DC1 | 10.1.0.11 | /32 | 10.1.0.11 -spine2-DC1 | 10.1.0.12 | /32 | 10.1.0.12 -spine3-DC1 | 10.1.0.13 | /32 | 10.1.0.13 -leaf1-DC1 | 10.1.0.21 | /32 | 10.1.0.21 -leaf2-DC1 | 10.1.0.22 | /32 | 10.1.0.22 -leaf3-DC1 | 10.1.0.23 | /32 | 10.1.0.23 -leaf4-DC1 | 10.1.0.24 | /32 | 10.1.0.24 -borderleaf1-DC1 | 10.1.0.31 | /32 | 10.1.0.31 -borderleaf2-DC1 | 10.1.0.32 | /32 | 10.1.0.32 +| Device | Loopback0 | Mask | Role | Router ID | +| --------------- | --------- | ---- | ---------- | --------- | +| spine1-DC1 | 10.1.0.11 | /32 | Spine | 10.1.0.11 | +| spine2-DC1 | 10.1.0.12 | /32 | Spine | 10.1.0.12 | +| spine3-DC1 | 10.1.0.13 | /32 | Spine | 10.1.0.13 | +| leaf1-DC1 | 10.1.0.21 | /32 | Leaf/Agg | 10.1.0.21 | +| leaf2-DC1 | 10.1.0.22 | /32 | Leaf/Agg | 10.1.0.22 | +| leaf3-DC1 | 10.1.0.23 | /32 | Leaf/Agg | 10.1.0.23 | +| leaf4-DC1 | 10.1.0.24 | /32 | Leaf/Agg | 10.1.0.24 | +| borderleaf1-DC1 | 10.1.0.31 | /32 | Border/DCI | 10.1.0.31 | +| borderleaf2-DC1 | 10.1.0.32 | /32 | Border/DCI | 10.1.0.32 | ``` -### Loopback1 Addresses (VTEP) +**Note**: Access switches are Layer 2 only - no loopbacks needed + +### Loopback1 Addresses (VTEP - Leaf Layer Only) **Subnet**: `10.1.1.0/24` ``` -Device | Loopback1 | Mask | Notes ---------------------|----------------|------|------------------------ -leaf1-DC1 | 10.1.1.21 | /32 | Shared with leaf2 -leaf2-DC1 | 10.1.1.21 | /32 | Shared with leaf1 (MLAG) -leaf3-DC1 | 10.1.1.23 | /32 | Shared with leaf4 -leaf4-DC1 | 10.1.1.23 | /32 | Shared with leaf3 (MLAG) -borderleaf1-DC1 | 10.1.1.31 | /32 | Shared with borderleaf2 -borderleaf2-DC1 | 10.1.1.31 | /32 | Shared with borderleaf1 (MLAG) +| Device | Loopback1 | Mask | Notes | +| --------------- | --------- | ---- | ------------------------------ | +| leaf1-DC1 | 10.1.1.21 | /32 | Shared with leaf2 | +| leaf2-DC1 | 10.1.1.21 | /32 | Shared with leaf1 (MLAG) | +| leaf3-DC1 | 10.1.1.23 | /32 | Shared with leaf4 | +| leaf4-DC1 | 10.1.1.23 | /32 | Shared with leaf3 (MLAG) | +| borderleaf1-DC1 | 10.1.1.31 | /32 | Shared with borderleaf2 | +| borderleaf2-DC1 | 10.1.1.31 | /32 | Shared with borderleaf1 (MLAG) | ``` -**Note**: Spines don't need Loopback1 (not VTEPs) +**Note**: +- Spines don't need Loopback1 (not VTEPs) +- Access switches don't need Loopback1 (L2 only) -### Point-to-Point Links (Spine-Leaf) +### Point-to-Point Links - Spine to Leaf (Underlay) **Subnet**: `10.1.10.0/24` (using /31 subnets) #### Spine1-DC1 Links ``` -Link | Leaf Side | Spine Side | Subnet -----------------------------------|--------------|--------------|-------- -leaf1-DC1:eth3 - spine1-DC1:eth2 | 10.1.10.0 | 10.1.10.1 | /31 -leaf2-DC1:eth3 - spine1-DC1:eth3 | 10.1.10.2 | 10.1.10.3 | /31 -leaf3-DC1:eth3 - spine1-DC1:eth4 | 10.1.10.4 | 10.1.10.5 | /31 -leaf4-DC1:eth3 - spine1-DC1:eth5 | 10.1.10.6 | 10.1.10.7 | /31 -border1-DC1:eth3 - spine1-DC1:eth6| 10.1.10.8 | 10.1.10.9 | /31 -border2-DC1:eth3 - spine1-DC1:eth7| 10.1.10.10 | 10.1.10.11 | /31 +| Link | Leaf Side | Spine Side | Subnet | +| ---------------------------------- | ---------- | ---------- | ------ | +| leaf1-DC1:eth3 - spine1-DC1:eth2 | 10.1.10.0 | 10.1.10.1 | /31 | +| leaf2-DC1:eth3 - spine1-DC1:eth3 | 10.1.10.2 | 10.1.10.3 | /31 | +| leaf3-DC1:eth3 - spine1-DC1:eth4 | 10.1.10.4 | 10.1.10.5 | /31 | +| leaf4-DC1:eth3 - spine1-DC1:eth5 | 10.1.10.6 | 10.1.10.7 | /31 | +| border1-DC1:eth3 - spine1-DC1:eth6 | 10.1.10.8 | 10.1.10.9 | /31 | +| border2-DC1:eth3 - spine1-DC1:eth7 | 10.1.10.10 | 10.1.10.11 | /31 | ``` #### Spine2-DC1 Links ``` -Link | Leaf Side | Spine Side | Subnet -----------------------------------|--------------|--------------|-------- -leaf1-DC1:eth4 - spine2-DC1:eth2 | 10.1.10.12 | 10.1.10.13 | /31 -leaf2-DC1:eth4 - spine2-DC1:eth3 | 10.1.10.14 | 10.1.10.15 | /31 -leaf3-DC1:eth4 - spine2-DC1:eth4 | 10.1.10.16 | 10.1.10.17 | /31 -leaf4-DC1:eth4 - spine2-DC1:eth5 | 10.1.10.18 | 10.1.10.19 | /31 -border1-DC1:eth4 - spine2-DC1:eth6| 10.1.10.20 | 10.1.10.21 | /31 -border2-DC1:eth4 - spine2-DC1:eth7| 10.1.10.22 | 10.1.10.23 | /31 +| Link | Leaf Side | Spine Side | Subnet | +| ---------------------------------- | ---------- | ---------- | ------ | +| leaf1-DC1:eth4 - spine2-DC1:eth2 | 10.1.10.12 | 10.1.10.13 | /31 | +| leaf2-DC1:eth4 - spine2-DC1:eth3 | 10.1.10.14 | 10.1.10.15 | /31 | +| leaf3-DC1:eth4 - spine2-DC1:eth4 | 10.1.10.16 | 10.1.10.17 | /31 | +| leaf4-DC1:eth4 - spine2-DC1:eth5 | 10.1.10.18 | 10.1.10.19 | /31 | +| border1-DC1:eth4 - spine2-DC1:eth6 | 10.1.10.20 | 10.1.10.21 | /31 | +| border2-DC1:eth4 - spine2-DC1:eth7 | 10.1.10.22 | 10.1.10.23 | /31 | ``` #### Spine3-DC1 Links ``` -Link | Leaf Side | Spine Side | Subnet -----------------------------------|--------------|--------------|-------- -leaf1-DC1:eth5 - spine3-DC1:eth2 | 10.1.10.24 | 10.1.10.25 | /31 -leaf2-DC1:eth5 - spine3-DC1:eth3 | 10.1.10.26 | 10.1.10.27 | /31 -leaf3-DC1:eth5 - spine3-DC1:eth4 | 10.1.10.28 | 10.1.10.29 | /31 -leaf4-DC1:eth5 - spine3-DC1:eth5 | 10.1.10.30 | 10.1.10.31 | /31 -border1-DC1:eth5 - spine3-DC1:eth6| 10.1.10.32 | 10.1.10.33 | /31 -border2-DC1:eth5 - spine3-DC1:eth7| 10.1.10.34 | 10.1.10.35 | /31 +| Link | Leaf Side | Spine Side | Subnet | +| ---------------------------------- | ---------- | ---------- | ------ | +| leaf1-DC1:eth5 - spine3-DC1:eth2 | 10.1.10.24 | 10.1.10.25 | /31 | +| leaf2-DC1:eth5 - spine3-DC1:eth3 | 10.1.10.26 | 10.1.10.27 | /31 | +| leaf3-DC1:eth5 - spine3-DC1:eth4 | 10.1.10.28 | 10.1.10.29 | /31 | +| leaf4-DC1:eth5 - spine3-DC1:eth5 | 10.1.10.30 | 10.1.10.31 | /31 | +| border1-DC1:eth5 - spine3-DC1:eth6 | 10.1.10.32 | 10.1.10.33 | /31 | +| border2-DC1:eth5 - spine3-DC1:eth7 | 10.1.10.34 | 10.1.10.35 | /31 | ``` +### Point-to-Point Links - Leaf to Access (L3 Uplinks) +**Subnet**: `10.1.20.0/24` (using /31 subnets) + +``` +| Link | Access Side | Leaf Side | Subnet | VLAN | +| --------------------------------- | ----------- | --------- | ------ | ---- | +| access1-DC1:eth1 - leaf1-DC1:eth7 | 10.1.20.0 | 10.1.20.1 | /31 | N/A | +| access1-DC1:eth2 - leaf2-DC1:eth7 | 10.1.20.2 | 10.1.20.3 | /31 | N/A | +| access2-DC1:eth1 - leaf3-DC1:eth7 | 10.1.20.4 | 10.1.20.5 | /31 | N/A | +| access2-DC1:eth2 - leaf4-DC1:eth7 | 10.1.20.6 | 10.1.20.7 | /31 | N/A | +``` + +**Note**: These can be L3 (routed) or L2 (trunk) depending on design choice + ### MLAG Peer Links (VLAN 4094) **Subnet**: `10.1.255.0/24` (using /30 subnets) ``` -MLAG Pair | Device | VLAN 4094 IP | Subnet ---------------------|-----------------|--------------|-------- -Leaf Pair 1 | leaf1-DC1 | 10.1.255.1 | /30 - | leaf2-DC1 | 10.1.255.2 | /30 -Leaf Pair 2 | leaf3-DC1 | 10.1.255.5 | /30 - | leaf4-DC1 | 10.1.255.6 | /30 -Border Leaf Pair | borderleaf1-DC1 | 10.1.255.9 | /30 - | borderleaf2-DC1 | 10.1.255.10 | /30 +| MLAG Pair | Device | VLAN 4094 IP | Subnet | +| ---------------- | --------------- | ------------ | ------ | +| Leaf Pair 1 | leaf1-DC1 | 10.1.255.1 | /30 | +| leaf2-DC1 | 10.1.255.2 | /30 | +| Leaf Pair 2 | leaf3-DC1 | 10.1.255.5 | /30 | +| leaf4-DC1 | 10.1.255.6 | /30 | +| Border Leaf Pair | borderleaf1-DC1 | 10.1.255.9 | /30 | +| borderleaf2-DC1 | 10.1.255.10 | /30 | ``` +**Note**: Access switches don't need MLAG (single ToR per bay) + --- ## 🏢 DC2 - Data Center 2 IP Plan -### Loopback0 Addresses (Router IDs) +### Loopback0 Addresses (Router IDs - Spine & Leaf Only) **Subnet**: `10.2.0.0/24` ``` -Device | Loopback0 | Mask | Router ID ---------------------|----------------|------|---------- -spine1-DC2 | 10.2.0.11 | /32 | 10.2.0.11 -spine2-DC2 | 10.2.0.12 | /32 | 10.2.0.12 -spine3-DC2 | 10.2.0.13 | /32 | 10.2.0.13 -leaf1-DC2 | 10.2.0.21 | /32 | 10.2.0.21 -leaf2-DC2 | 10.2.0.22 | /32 | 10.2.0.22 -leaf3-DC2 | 10.2.0.23 | /32 | 10.2.0.23 -leaf4-DC2 | 10.2.0.24 | /32 | 10.2.0.24 -borderleaf1-DC2 | 10.2.0.31 | /32 | 10.2.0.31 -borderleaf2-DC2 | 10.2.0.32 | /32 | 10.2.0.32 +| Device | Loopback0 | Mask | Role | Router ID | +| --------------- | --------- | ---- | ---------- | --------- | +| spine1-DC2 | 10.2.0.11 | /32 | Spine | 10.2.0.11 | +| spine2-DC2 | 10.2.0.12 | /32 | Spine | 10.2.0.12 | +| spine3-DC2 | 10.2.0.13 | /32 | Spine | 10.2.0.13 | +| leaf1-DC2 | 10.2.0.21 | /32 | Leaf/Agg | 10.2.0.21 | +| leaf2-DC2 | 10.2.0.22 | /32 | Leaf/Agg | 10.2.0.22 | +| leaf3-DC2 | 10.2.0.23 | /32 | Leaf/Agg | 10.2.0.23 | +| leaf4-DC2 | 10.2.0.24 | /32 | Leaf/Agg | 10.2.0.24 | +| borderleaf1-DC2 | 10.2.0.31 | /32 | Border/DCI | 10.2.0.31 | +| borderleaf2-DC2 | 10.2.0.32 | /32 | Border/DCI | 10.2.0.32 | ``` -### Loopback1 Addresses (VTEP) +### Loopback1 Addresses (VTEP - Leaf Layer Only) **Subnet**: `10.2.1.0/24` ``` -Device | Loopback1 | Mask | Notes ---------------------|----------------|------|------------------------ -leaf1-DC2 | 10.2.1.21 | /32 | Shared with leaf2 -leaf2-DC2 | 10.2.1.21 | /32 | Shared with leaf1 (MLAG) -leaf3-DC2 | 10.2.1.23 | /32 | Shared with leaf4 -leaf4-DC2 | 10.2.1.23 | /32 | Shared with leaf3 (MLAG) -borderleaf1-DC2 | 10.2.1.31 | /32 | Shared with borderleaf2 -borderleaf2-DC2 | 10.2.1.31 | /32 | Shared with borderleaf1 (MLAG) +| Device | Loopback1 | Mask | Notes | +| --------------- | --------- | ---- | ------------------------------ | +| leaf1-DC2 | 10.2.1.21 | /32 | Shared with leaf2 | +| leaf2-DC2 | 10.2.1.21 | /32 | Shared with leaf1 (MLAG) | +| leaf3-DC2 | 10.2.1.23 | /32 | Shared with leaf4 | +| leaf4-DC2 | 10.2.1.23 | /32 | Shared with leaf3 (MLAG) | +| borderleaf1-DC2 | 10.2.1.31 | /32 | Shared with borderleaf2 | +| borderleaf2-DC2 | 10.2.1.31 | /32 | Shared with borderleaf1 (MLAG) | ``` -### Point-to-Point Links (Spine-Leaf) +### Point-to-Point Links - Spine to Leaf (Underlay) **Subnet**: `10.2.10.0/24` (using /31 subnets) **Same pattern as DC1**, but using `10.2.10.x` range: @@ -216,18 +246,30 @@ borderleaf2-DC2 | 10.2.1.31 | /32 | Shared with borderleaf1 (MLAG) - Spine2 links: 10.2.10.12 - 10.2.10.23 - Spine3 links: 10.2.10.24 - 10.2.10.35 +### Point-to-Point Links - Leaf to Access (L3 Uplinks) +**Subnet**: `10.2.20.0/24` (using /31 subnets) + +``` +| Link | Access Side | Leaf Side | Subnet | +| --------------------------------- | ----------- | --------- | ------ | +| access1-DC2:eth1 - leaf1-DC2:eth7 | 10.2.20.0 | 10.2.20.1 | /31 | +| access1-DC2:eth2 - leaf2-DC2:eth7 | 10.2.20.2 | 10.2.20.3 | /31 | +| access2-DC2:eth1 - leaf3-DC2:eth7 | 10.2.20.4 | 10.2.20.5 | /31 | +| access2-DC2:eth2 - leaf4-DC2:eth7 | 10.2.20.6 | 10.2.20.7 | /31 | +``` + ### MLAG Peer Links (VLAN 4094) **Subnet**: `10.2.255.0/24` (using /30 subnets) ``` -MLAG Pair | Device | VLAN 4094 IP | Subnet ---------------------|-----------------|--------------|-------- -Leaf Pair 1 | leaf1-DC2 | 10.2.255.1 | /30 - | leaf2-DC2 | 10.2.255.2 | /30 -Leaf Pair 2 | leaf3-DC2 | 10.2.255.5 | /30 - | leaf4-DC2 | 10.2.255.6 | /30 -Border Leaf Pair | borderleaf1-DC2 | 10.2.255.9 | /30 - | borderleaf2-DC2 | 10.2.255.10 | /30 +| MLAG Pair | Device | VLAN 4094 IP | Subnet | +| ---------------- | --------------- | ------------ | ------ | +| Leaf Pair 1 | leaf1-DC2 | 10.2.255.1 | /30 | +| leaf2-DC2 | 10.2.255.2 | /30 | +| Leaf Pair 2 | leaf3-DC2 | 10.2.255.5 | /30 | +| leaf4-DC2 | 10.2.255.6 | /30 | +| Border Leaf Pair | borderleaf1-DC2 | 10.2.255.9 | /30 | +| borderleaf2-DC2 | 10.2.255.10 | /30 | ``` --- @@ -236,21 +278,21 @@ Border Leaf Pair | borderleaf1-DC2 | 10.2.255.9 | /30 ### DCI Loopback ``` -Device | Loopback0 | Mask | Router ID ---------------------|----------------|------|---------- -DCI | 10.253.0.1 | /32 | 10.253.0.1 +| Device | Loopback0 | Mask | Router ID | +| ------ | ---------- | ---- | ---------- | +| DCI | 10.253.0.1 | /32 | 10.253.0.1 | ``` ### DCI Point-to-Point Links **Subnet**: `10.253.254.0/24` (using /31 subnets) ``` -Link | Borderleaf Side | DCI Side | Subnet -------------------------------------|-----------------|--------------|-------- -borderleaf1-DC1:eth12 - DCI:eth1 | 10.253.254.0 | 10.253.254.1 | /31 -borderleaf2-DC1:eth12 - DCI:eth2 | 10.253.254.2 | 10.253.254.3 | /31 -borderleaf1-DC2:eth12 - DCI:eth3 | 10.253.254.4 | 10.253.254.5 | /31 -borderleaf2-DC2:eth12 - DCI:eth4 | 10.253.254.6 | 10.253.254.7 | /31 +| Link | Borderleaf Side | DCI Side | Subnet | +| -------------------------------- | --------------- | ------------ | ------ | +| borderleaf1-DC1:eth12 - DCI:eth1 | 10.253.254.0 | 10.253.254.1 | /31 | +| borderleaf2-DC1:eth12 - DCI:eth2 | 10.253.254.2 | 10.253.254.3 | /31 | +| borderleaf1-DC2:eth12 - DCI:eth3 | 10.253.254.4 | 10.253.254.5 | /31 | +| borderleaf2-DC2:eth12 - DCI:eth4 | 10.253.254.6 | 10.253.254.7 | /31 | ``` --- @@ -261,22 +303,22 @@ borderleaf2-DC2:eth12 - DCI:eth4 | 10.253.254.6 | 10.253.254.7 | /31 **Subnet**: `172.16.x.0/24` (one /24 per VLAN) ``` -VLAN ID | VLAN Name | Subnet | Gateway | Purpose ---------|--------------|-----------------|--------------|------------------ -100 | TENANT-A | 172.16.100.0/24 | 172.16.100.1 | Tenant A workloads -200 | TENANT-B | 172.16.200.0/24 | 172.16.200.1 | Tenant B workloads -300 | DMZ | 172.16.300.0/24 | 172.16.300.1 | DMZ services -4094 | MLAG-PEER | (see above) | N/A | MLAG peer link +| VLAN ID | VLAN Name | Subnet | Gateway | Purpose | +| ------- | --------- | --------------- | ------------ | ------------------ | +| 100 | TENANT-A | 172.16.100.0/24 | 172.16.100.1 | Tenant A workloads | +| 200 | TENANT-B | 172.16.200.0/24 | 172.16.200.1 | Tenant B workloads | +| 300 | DMZ | 172.16.300.0/24 | 172.16.300.1 | DMZ services | +| 4094 | MLAG-PEER | (see above) | N/A | MLAG peer link | ``` ### Host IP Assignments ``` -Host Device | VLAN | IP Address | Gateway | Bond Interface ----------------|------|-----------------|--------------|--------------- -host1-DC1 | 100 | 172.16.100.10/24| 172.16.100.1 | bond0 -host2-DC1 | 200 | 172.16.200.10/24| 172.16.200.1 | bond0 -host1-DC2 | 100 | 172.16.100.20/24| 172.16.100.1 | bond0 -host2-DC2 | 200 | 172.16.200.20/24| 172.16.200.1 | bond0 +| Host Device | Bay | VLAN | IP Address | Gateway | Access Switch | +| ----------- | --- | ---- | ---------------- | ------------ | ------------- | +| host1-DC1 | 1 | 100 | 172.16.100.10/24 | 172.16.100.1 | access1-DC1 | +| host2-DC1 | 2 | 200 | 172.16.200.10/24 | 172.16.200.1 | access2-DC1 | +| host1-DC2 | 1 | 100 | 172.16.100.20/24 | 172.16.100.1 | access1-DC2 | +| host2-DC2 | 2 | 200 | 172.16.200.20/24 | 172.16.200.1 | access2-DC2 | ``` --- @@ -285,52 +327,84 @@ host2-DC2 | 200 | 172.16.200.20/24| 172.16.200.1 | bond0 ### DC1 ASNs ``` -Device Type | Devices | ASN ---------------------|------------------------------|-------- -Spines | spine1-3 DC1 | 65100 -Leaf Pair 1 | leaf1-DC1, leaf2-DC1 | 65101 -Leaf Pair 2 | leaf3-DC1, leaf4-DC1 | 65102 -Border Leaf Pair | borderleaf1-2 DC1 | 65103 +| Device Type | ASN | Devices | BGP Role | +| ---------------- | ----- | ----------------- | ------------------ | +| Spines | 65100 | spine1-3 DC1 | eBGP to leafs | +| Leaf Pair 1 | 65101 | leaf1-2 DC1 | eBGP to spines | +| Leaf Pair 2 | 65102 | leaf3-4 DC1 | eBGP to spines | +| Border Leaf Pair | 65103 | borderleaf1-2 DC1 | eBGP to spines+DCI | +| Access Switches | N/A | access1-2 DC1 | L2 only (no BGP) | ``` ### DC2 ASNs ``` -Device Type | Devices | ASN ---------------------|------------------------------|-------- -Spines | spine1-3 DC2 | 65200 -Leaf Pair 1 | leaf1-DC2, leaf2-DC2 | 65201 -Leaf Pair 2 | leaf3-DC2, leaf4-DC2 | 65202 -Border Leaf Pair | borderleaf1-2 DC2 | 65203 +| Device Type | ASN | Devices | BGP Role | +| ---------------- | ----- | ----------------- | ------------------ | +| Spines | 65200 | spine1-3 DC2 | eBGP to leafs | +| Leaf Pair 1 | 65201 | leaf1-2 DC2 | eBGP to spines | +| Leaf Pair 2 | 65202 | leaf3-4 DC2 | eBGP to spines | +| Border Leaf Pair | 65203 | borderleaf1-2 DC2 | eBGP to spines+DCI | +| Access Switches | N/A | access1-2 DC2 | L2 only (no BGP) | ``` ### DCI ASN ``` -Device | ASN | Purpose ---------------------|-------|-------------------------------- -DCI | 65000 | Neutral AS for inter-DC routing +| Device | ASN | Purpose | +| ------ | ----- | ------------------------------- | +| DCI | 65000 | Neutral AS for inter-DC routing | ``` --- -## 🔍 IP Address Validation +## 🏗️ Architecture Benefits -### No Conflicts With Your LAN ✅ +### Why Access Layer? + +1. **Scalability**: Each bay/rack gets its own switch + - Easy to add more bays: just add another access switch + - Leaf ports don't run out (48 port switch = 24 racks possible) + +2. **Simplified Cabling**: + - Hosts only need 1 cable (to local access switch) + - Access switch dual-homes to MLAG leaf pair + - Reduces cross-rack cabling complexity + +3. **Automation-Friendly**: + - Access switches are identical (same config template) + - Only variables: hostname, bay number, VLANs + - Easy to generate configs from data model + +4. **Cost-Effective**: + - Access switches can be cheaper models (L2/L3 basic) + - Leafs reserved for VXLAN/EVPN (more expensive) + +5. **Failure Domain Isolation**: + - Bay/rack failure contained to access switch + - Doesn't affect underlay routing + +### Automation Implications + +**For Infrahub Data Model**: ``` -Your LAN: 192.168.1.0/24 -Our Plan: 10.x.x.x and 172.16.x.x -Conflict Risk: NONE ✅ +Site (DC1) +├── Pod (Fabric-1) +│ ├── Spine Layer +│ │ ├── spine1-DC1 +│ │ ├── spine2-DC1 +│ │ └── spine3-DC1 +│ ├── Leaf Layer +│ │ ├── MLAG Pair 1 (leaf1-2) +│ │ └── MLAG Pair 2 (leaf3-4) +│ └── Access Layer +│ ├── Bay 1 → access1-DC1 +│ └── Bay 2 → access2-DC1 ``` -### Subnets Summary -``` -Purpose | Subnet Range | Total IPs ----------------------|-------------------|---------- -Management | 10.255.0.0/24 | 254 -DC1 Infrastructure | 10.1.0.0/16 | 65,534 -DC2 Infrastructure | 10.2.0.0/16 | 65,534 -DCI | 10.253.0.0/16 | 65,534 -Tenant Networks | 172.16.0.0/16 | 65,534 -``` +**Templating Strategy**: +- **Spine**: 1 template (all identical except IPs) +- **Leaf**: 2 templates (odd/even for MLAG) +- **Access**: 1 template (all identical except bay# and VLANs) +- **Variables**: Site, bay_number, VLANs, uplink_ips --- @@ -338,34 +412,26 @@ Tenant Networks | 172.16.0.0/16 | 65,534 ### Test Management Connectivity ```bash -# DC1 Spines -ping 10.255.0.11 -ping 10.255.0.12 -ping 10.255.0.13 +# DC1 Infrastructure +ping 10.255.0.11 # spine1-DC1 +ping 10.255.0.21 # leaf1-DC1 +ping 10.255.0.71 # access1-DC1 -# DC1 Leafs -ping 10.255.0.21 -ping 10.255.0.22 +# DC2 Infrastructure +ping 10.255.0.41 # spine1-DC2 +ping 10.255.0.51 # leaf1-DC2 +ping 10.255.0.81 # access1-DC2 # DCI -ping 10.255.0.50 +ping 10.255.0.100 + +# Hosts +ping 10.255.0.201 # host1-DC1 ``` -### SSH Access +### Verify Underlay from Leaf ```bash -# Access spine1-DC1 -ssh admin@10.255.0.11 - -# Access leaf1-DC1 -ssh admin@10.255.0.21 - -# Access DCI -ssh admin@10.255.0.50 -``` - -### Verify Underlay from any leaf -```bash -# Check loopback reachability (should reach all loopbacks) +# Check loopback reachability ping 10.1.0.11 source 10.1.0.21 # Check BGP neighbors @@ -373,20 +439,33 @@ show ip bgp summary # Verify ECMP paths show ip route 10.1.1.23 + +# Check VXLAN tunnels +show vxlan vtep +show vxlan address-table +``` + +### Verify Access Layer Connectivity +```bash +# From access switch +show lldp neighbors +show port-channel summary + +# From leaf (check access uplinks) +show interfaces ethernet 7 status +show interfaces ethernet 7 description ``` --- ## 💡 Design Highlights -1. **Clean Separation**: Each DC has its own /16 (10.1.x.x and 10.2.x.x) -2. **Consistent Numbering**: Same pattern in both DCs -3. **Easy Troubleshooting**: IP tells you the location/role - - `.0.x` = Loopback0 (router-id) - - `.1.x` = Loopback1 (VTEP) - - `.10.x` = P2P links - - `.255.x` = MLAG peer links -4. **Scalable**: Room for growth in each range -5. **No LAN Conflicts**: Completely avoids 192.168.x.x space +1. **4-Tier Architecture**: Spine → Leaf → Access → Host +2. **Clean Separation**: Each layer has distinct role +3. **MLAG at Leaf Layer**: Access switches dual-home to leaf pairs +4. **Access Layer is L2**: Simplifies config, VLANs stretched via VXLAN +5. **One Access Per Bay**: Realistic rack/bay topology +6. **Automation-Ready**: Consistent patterns, easy templating +7. **Scalable**: Add bays by adding access switches -This addressing plan is production-ready and automation-friendly! 🎯 \ No newline at end of file +This design is **production-ready** and **automation-optimized**! 🎯 \ No newline at end of file diff --git a/containerlab/arista-l5-topology.clab.yml b/containerlab/arista-l5-topology.clab.yml index f3b7eba..e7d36b9 100644 --- a/containerlab/arista-l5-topology.clab.yml +++ b/containerlab/arista-l5-topology.clab.yml @@ -29,7 +29,7 @@ topology: # startup-config: configs/spine3-DC1.cfg # ========================================== - # DC1 - LEAF LAYER + # DC1 - LEAF LAYER (Aggregation/Distribution) # ========================================== leaf1-DC1: kind: arista_ceos @@ -52,7 +52,7 @@ topology: # startup-config: configs/leaf4-DC1.cfg # ========================================== - # DC1 - BORDER LEAF LAYER + # DC1 - BORDER LEAF LAYER (DCI Gateway) # ========================================== borderleaf1-DC1: kind: arista_ceos @@ -64,6 +64,20 @@ topology: mgmt-ipv4: 10.255.0.32 # startup-config: configs/borderleaf2-DC1.cfg + # ========================================== + # DC1 - ACCESS LAYER (Rack/Bay Switches) + # Each bay/rack gets 1 access switch + # ========================================== + access1-DC1: + kind: arista_ceos + mgmt-ipv4: 10.255.0.71 + # startup-config: configs/access1-DC1.cfg + + access2-DC1: + kind: arista_ceos + mgmt-ipv4: 10.255.0.72 + # startup-config: configs/access2-DC1.cfg + # ========================================== # DC2 - SPINE LAYER # ========================================== @@ -83,7 +97,7 @@ topology: # startup-config: configs/spine3-DC2.cfg # ========================================== - # DC2 - LEAF LAYER + # DC2 - LEAF LAYER (Aggregation/Distribution) # ========================================== leaf1-DC2: kind: arista_ceos @@ -106,7 +120,7 @@ topology: # startup-config: configs/leaf4-DC2.cfg # ========================================== - # DC2 - BORDER LEAF LAYER + # DC2 - BORDER LEAF LAYER (DCI Gateway) # ========================================== borderleaf1-DC2: kind: arista_ceos @@ -118,6 +132,20 @@ topology: mgmt-ipv4: 10.255.0.62 # startup-config: configs/borderleaf2-DC2.cfg + # ========================================== + # DC2 - ACCESS LAYER (Rack/Bay Switches) + # Each bay/rack gets 1 access switch + # ========================================== + access1-DC2: + kind: arista_ceos + mgmt-ipv4: 10.255.0.81 + # startup-config: configs/access1-DC2.cfg + + access2-DC2: + kind: arista_ceos + mgmt-ipv4: 10.255.0.82 + # startup-config: configs/access2-DC2.cfg + # ========================================== # DCI SWITCH (Data Center Interconnect) # ========================================== @@ -127,68 +155,36 @@ topology: # startup-config: configs/DCI.cfg # ========================================== - # HOST DEVICES - # Using Linux containers as hosts + # HOST DEVICES - DC1 + # Connected to access switches (1 per rack/bay) # ========================================== host1-DC1: kind: linux mgmt-ipv4: 10.255.0.201 image: alpine:latest - exec: - - ip link add bond0 type bond mode 802.3ad - - ip link set eth1 down - - ip link set eth2 down - - ip link set eth1 master bond0 - - ip link set eth2 master bond0 - - ip link set bond0 up - - ip link set eth1 up - - ip link set eth2 up host2-DC1: kind: linux mgmt-ipv4: 10.255.0.202 image: alpine:latest - exec: - - ip link add bond0 type bond mode 802.3ad - - ip link set eth1 down - - ip link set eth2 down - - ip link set eth1 master bond0 - - ip link set eth2 master bond0 - - ip link set bond0 up - - ip link set eth1 up - - ip link set eth2 up + # ========================================== + # HOST DEVICES - DC2 + # Connected to access switches (1 per rack/bay) + # ========================================== host1-DC2: kind: linux mgmt-ipv4: 10.255.0.211 image: alpine:latest - exec: - - ip link add bond0 type bond mode 802.3ad - - ip link set eth1 down - - ip link set eth2 down - - ip link set eth1 master bond0 - - ip link set eth2 master bond0 - - ip link set bond0 up - - ip link set eth1 up - - ip link set eth2 up host2-DC2: kind: linux mgmt-ipv4: 10.255.0.212 image: alpine:latest - exec: - - ip link add bond0 type bond mode 802.3ad - - ip link set eth1 down - - ip link set eth2 down - - ip link set eth1 master bond0 - - ip link set eth2 master bond0 - - ip link set bond0 up - - ip link set eth1 up - - ip link set eth2 up links: # ========================================== - # DC1 - SPINE to LEAF Connections + # DC1 - SPINE to LEAF Connections (Underlay) # ========================================== # Spine1-DC1 to leafs @@ -216,7 +212,7 @@ topology: - endpoints: ["spine3-DC1:eth7", "borderleaf2-DC1:eth5"] # ========================================== - # DC1 - MLAG Peer Links + # DC1 - MLAG Peer Links (Leaf Layer) # ========================================== - endpoints: ["leaf1-DC1:eth1", "leaf2-DC1:eth1"] - endpoints: ["leaf1-DC1:eth2", "leaf2-DC1:eth2"] @@ -228,7 +224,27 @@ topology: - endpoints: ["borderleaf1-DC1:eth2", "borderleaf2-DC1:eth2"] # ========================================== - # DC2 - SPINE to LEAF Connections + # DC1 - LEAF to ACCESS Connections + # Access switches dual-homed to MLAG leaf pairs + # ========================================== + + # access1-DC1 connects to leaf1-DC1 and leaf2-DC1 (MLAG) + - endpoints: ["leaf1-DC1:eth7", "access1-DC1:eth1"] + - endpoints: ["leaf2-DC1:eth7", "access1-DC1:eth2"] + + # access2-DC1 connects to leaf3-DC1 and leaf4-DC1 (MLAG) + - endpoints: ["leaf3-DC1:eth7", "access2-DC1:eth1"] + - endpoints: ["leaf4-DC1:eth7", "access2-DC1:eth2"] + + # ========================================== + # DC1 - ACCESS to HOST Connections + # Hosts connect to their local access switch + # ========================================== + - endpoints: ["access1-DC1:eth10", "host1-DC1:eth1"] + - endpoints: ["access2-DC1:eth10", "host2-DC1:eth1"] + + # ========================================== + # DC2 - SPINE to LEAF Connections (Underlay) # ========================================== # Spine1-DC2 to leafs @@ -256,7 +272,7 @@ topology: - endpoints: ["spine3-DC2:eth7", "borderleaf2-DC2:eth5"] # ========================================== - # DC2 - MLAG Peer Links + # DC2 - MLAG Peer Links (Leaf Layer) # ========================================== - endpoints: ["leaf1-DC2:eth1", "leaf2-DC2:eth1"] - endpoints: ["leaf1-DC2:eth2", "leaf2-DC2:eth2"] @@ -267,30 +283,30 @@ topology: - endpoints: ["borderleaf1-DC2:eth1", "borderleaf2-DC2:eth1"] - endpoints: ["borderleaf1-DC2:eth2", "borderleaf2-DC2:eth2"] + # ========================================== + # DC2 - LEAF to ACCESS Connections + # Access switches dual-homed to MLAG leaf pairs + # ========================================== + + # access1-DC2 connects to leaf1-DC2 and leaf2-DC2 (MLAG) + - endpoints: ["leaf1-DC2:eth7", "access1-DC2:eth1"] + - endpoints: ["leaf2-DC2:eth7", "access1-DC2:eth2"] + + # access2-DC2 connects to leaf3-DC2 and leaf4-DC2 (MLAG) + - endpoints: ["leaf3-DC2:eth7", "access2-DC2:eth1"] + - endpoints: ["leaf4-DC2:eth7", "access2-DC2:eth2"] + + # ========================================== + # DC2 - ACCESS to HOST Connections + # Hosts connect to their local access switch + # ========================================== + - endpoints: ["access1-DC2:eth10", "host1-DC2:eth1"] + - endpoints: ["access2-DC2:eth10", "host2-DC2:eth1"] + # ========================================== # DCI Links (Border Leaf to DCI) # ========================================== - endpoints: ["borderleaf1-DC1:eth12", "DCI:eth1"] - endpoints: ["borderleaf2-DC1:eth12", "DCI:eth2"] - endpoints: ["borderleaf1-DC2:eth12", "DCI:eth3"] - - endpoints: ["borderleaf2-DC2:eth12", "DCI:eth4"] - - # ========================================== - # Host Connections (MLAG dual-homed) - # ========================================== - - # Host1-DC1 to leaf1-DC1 and leaf2-DC1 - - endpoints: ["leaf1-DC1:eth7", "host1-DC1:eth1"] - - endpoints: ["leaf2-DC1:eth7", "host1-DC1:eth2"] - - # Host2-DC1 to leaf3-DC1 and leaf4-DC1 - - endpoints: ["leaf3-DC1:eth7", "host2-DC1:eth1"] - - endpoints: ["leaf4-DC1:eth7", "host2-DC1:eth2"] - - # Host1-DC2 to leaf1-DC2 and leaf2-DC2 - - endpoints: ["leaf1-DC2:eth7", "host1-DC2:eth1"] - - endpoints: ["leaf2-DC2:eth7", "host1-DC2:eth2"] - - # Host2-DC2 to leaf3-DC2 and leaf4-DC2 - - endpoints: ["leaf3-DC2:eth7", "host2-DC2:eth1"] - - endpoints: ["leaf4-DC2:eth7", "host2-DC2:eth2"] \ No newline at end of file + - endpoints: ["borderleaf2-DC2:eth12", "DCI:eth4"] \ No newline at end of file