infra-clab-pipeline/infrahub/design.md

# Data Center Design Philosophy & Automation Model

## 🎯 Core Concept: Datacenter as a Generator

### The Vision
A **Datacenter** is a **generator object** in Infrahub that creates all child objects automatically based on parent attributes and business logic.

```
Input: Datacenter Object (with attributes)
  ↓
Business Logic (naming conventions, network design patterns)
  ↓
Output: Complete infrastructure (devices, interfaces, IPs, BGP sessions)
```

---

## 📋 Hierarchical Data Model

### Level 0: Organization (Root)
```
Organization
├── Sites (geographic locations)
└── IP Address Space (RFC 1918 allocations)
```

### Level 1: Site (Geographic)
```
Site
├── Name: "Paris-DC1"
├── Location: "Paris, France"
├── Type: "Production"
└── Parent Subnet: 10.0.0.0/8
```

### Level 2: Datacenter (Logical Fabric)
```
Datacenter
├── Site: Paris-DC1
├── DC ID: 1 (for IP addressing: 10.1.x.x)
├── Number of Bays: 2 (scalable)
├── Spine Count: 3 (fixed for now)
├── Border Leaf Count: 2 (fixed for now)
└── Generates:
    ├── Devices (Spines, Leafs, Border Leafs, Access switches)
    ├── IP Subnets (loopbacks, P2P, MLAG, tenants)
    ├── Interfaces (physical, loopbacks, VLANs)
    ├── BGP Configuration (ASNs, peers)
    └── MLAG Pairs
```

### Level 3: Bay/Rack (Physical)
```
Bay
├── Bay ID: 1, 2, 3...
├── Access Switch: access{bay_id}-DC{dc_id}
├── Leaf Pair: Assigned based on bay_id (round-robin)
├── Host Count: Variable (1-48 per bay)
└── Generates:
    ├── Access Switch device
    ├── Access-to-Leaf connections
    ├── Host connections
    └── VLAN assignments
```

### Level 4: Devices (Auto-generated)
```
Device
├── Hostname: (from naming convention)
├── Role: spine|leaf|borderleaf|access
├── Interfaces: (auto-generated based on role)
├── IP Addresses: (auto-allocated from parent subnets)
├── BGP Config: (auto-generated based on role)
└── MLAG Config: (if part of pair)
```

---

## 🏗️ Datacenter Generator - Input Attributes

### Required Attributes
```yaml
name: "DC1"                          # Human-readable name
dc_id: 1                             # Numeric ID (for IP addressing)
site: "Paris-DC1"                    # Parent site reference
parent_subnet: "10.0.0.0/8"          # Address space allocation
number_of_bays: 2                    # Initial bay count (scalable)
has_border_leafs: true               # Include border leafs for DCI capability
```

### Optional Attributes (with defaults)
```yaml
spine_count: 3                       # Default: 3
spine_asn: 65100                     # Default: 65000 + (dc_id * 100)
border_leaf_count: 2                 # Default: 2 (0 if has_border_leafs: false)
bgp_base_asn: 65000                  # Default: 65000
mlag_domain_id: "MLAG"               # Default: "MLAG"
mtu: 9214                            # Default: 9214
dci_enabled: false                   # Default: false (configure eth12 as shutdown)
```

### Derived/Computed Attributes (auto-calculated)
```yaml
# Computed from dc_id
loopback0_subnet: "10.{dc_id}.0.0/24"
loopback1_subnet: "10.{dc_id}.1.0/24"
spine_leaf_p2p_subnet: "10.{dc_id}.10.0/24"
leaf_access_p2p_subnet: "10.{dc_id}.20.0/24"
mlag_peer_subnet: "10.{dc_id}.255.0/24"

# Computed from number_of_bays
leaf_pair_count: ceil(number_of_bays / 2)  # 2 bays per leaf pair
total_leaf_count: leaf_pair_count * 2
total_access_count: number_of_bays
```

---

## 🎨 Business Logic Rules

### 1. Naming Conventions

#### Spine Switches
```
Pattern: spine{spine_number}-DC{dc_id}
Example: spine1-DC1, spine2-DC1, spine3-DC1
```

#### Leaf Switches
```
Pattern: leaf{leaf_number}-DC{dc_id}
Example: leaf1-DC1, leaf2-DC1, leaf3-DC1, leaf4-DC1
```

#### Border Leaf Switches
```
Pattern: borderleaf{number}-DC{dc_id}
Example: borderleaf1-DC1, borderleaf2-DC1
```

#### Access Switches
```
Pattern: access{bay_id}-DC{dc_id}
Example: access1-DC1, access2-DC1
```

#### Hosts
```
Pattern: host{host_number}-DC{dc_id}
Example: host1-DC1, host2-DC1
```

### 2. Bay-to-Leaf Assignment Logic

**Rule**: Each bay connects to ONE leaf pair (round-robin assignment)

```python
# Pseudo-code
leaf_pairs = [
    [leaf1, leaf2],    # MLAG Pair 1
    [leaf3, leaf4],    # MLAG Pair 2
    [leaf5, leaf6],    # MLAG Pair 3 (if exists)
]

def assign_bay_to_leaf_pair(bay_id, leaf_pairs):
    pair_index = (bay_id - 1) // 2  # Integer division
    return leaf_pairs[pair_index]

# Examples:
# bay_id=1 → leaf_pairs[0] → [leaf1, leaf2]
# bay_id=2 → leaf_pairs[0] → [leaf1, leaf2]
# bay_id=3 → leaf_pairs[1] → [leaf3, leaf4]
# bay_id=4 → leaf_pairs[1] → [leaf3, leaf4]
```

**Result**: 
- Bay 1 & 2 → Leaf Pair 1 (leaf1-2)
- Bay 3 & 4 → Leaf Pair 2 (leaf3-4)
- Bay 5 & 6 → Leaf Pair 3 (leaf5-6)

### 3. MLAG Pairing Logic

**Rule**: Odd-numbered leafs pair with next even-numbered leaf

```python
def create_mlag_pairs(total_leafs):
    pairs = []
    for i in range(1, total_leafs + 1, 2):
        pairs.append([f"leaf{i}", f"leaf{i+1}"])
    return pairs

# Example with 4 leafs:
# → [[leaf1, leaf2], [leaf3, leaf4]]
```

### 4. Interface Assignment Logic

#### Spine Interfaces
```
Role: Connect to all leafs + border leafs
Pattern:
  eth2 → leaf1
  eth3 → leaf2
  eth4 → leaf3
  eth5 → leaf4
  eth6 → borderleaf1
  eth7 → borderleaf2
```

#### Leaf Interfaces
```
Role: Uplinks to spines, MLAG peer, downlinks to access
Pattern:
  eth1-2  → MLAG peer link (to paired leaf)
  eth3-5  → Uplinks (spine1, spine2, spine3)
  eth7+   → Downlinks to access switches (2 bays per pair)
```

#### Access Interfaces
```
Role: Uplinks to leaf pair, downlinks to hosts
Pattern:
  eth1-2  → Uplinks (to leaf pair via MLAG)
  eth10+  → Host connections
```

### 5. IP Address Allocation Logic

#### Management IPs (10.255.0.0/24)
```python
def allocate_mgmt_ip(device_role, device_number, dc_id):
    base_octets = {
        'spine': 10 + (dc_id - 1) * 30,      # DC1: 11-13, DC2: 41-43
        'leaf': 20 + (dc_id - 1) * 30,       # DC1: 21-24, DC2: 51-54
        'borderleaf': 30 + (dc_id - 1) * 30, # DC1: 31-32, DC2: 61-62
        'access': 70 + (dc_id - 1) * 10,     # DC1: 71-72, DC2: 81-82
        'host': 200 + (dc_id - 1) * 10,      # DC1: 201-202, DC2: 211-212
    }
    return f"10.255.0.{base_octets[device_role] + device_number - 1}"

# Examples:
# spine1, DC1 → 10.255.0.11
# leaf1, DC1  → 10.255.0.21
# access1, DC2 → 10.255.0.81
```

#### Loopback0 (Router IDs)
```python
def allocate_loopback0(device_role, device_number, dc_id):
    role_base = {
        'spine': 11,
        'leaf': 21,
        'borderleaf': 31,
    }
    return f"10.{dc_id}.0.{role_base[device_role] + device_number - 1}/32"

# Examples:
# spine1, DC1 → 10.1.0.11/32
# leaf3, DC1  → 10.1.0.23/32
# spine2, DC2 → 10.2.0.12/32
```

#### Loopback1 (VTEP - Shared by MLAG pairs)
```python
def allocate_loopback1_vtep(leaf_number, dc_id):
    # Odd and even leafs share same VTEP
    vtep_base = ((leaf_number - 1) // 2) * 2 + 21
    return f"10.{dc_id}.1.{vtep_base}/32"

# Examples:
# leaf1, DC1 → 10.1.1.21/32 (shared with leaf2)
# leaf2, DC1 → 10.1.1.21/32 (shared with leaf1)
# leaf3, DC1 → 10.1.1.23/32 (shared with leaf4)
# leaf4, DC1 → 10.1.1.23/32 (shared with leaf3)
```

#### P2P Links (Spine-Leaf)
```python
def allocate_spine_leaf_p2p(spine_num, leaf_num, dc_id):
    # Each spine gets 6 leafs (including borderleafs)
    # Each link uses /31 (2 IPs)
    offset = (spine_num - 1) * 12 + (leaf_num - 1) * 2
    leaf_ip = f"10.{dc_id}.10.{offset}"
    spine_ip = f"10.{dc_id}.10.{offset + 1}"
    return (leaf_ip, spine_ip)

# Example:
# spine1 → leaf1 → (10.1.10.0, 10.1.10.1)
# spine1 → leaf2 → (10.1.10.2, 10.1.10.3)
```

#### P2P Links (Leaf-Access)
```python
def allocate_leaf_access_p2p(access_num, leaf_num, dc_id):
    # 2 uplinks per access (to MLAG leaf pair)
    offset = (access_num - 1) * 4 + (leaf_num % 2) * 2
    access_ip = f"10.{dc_id}.20.{offset}"
    leaf_ip = f"10.{dc_id}.20.{offset + 1}"
    return (access_ip, leaf_ip)

# Example:
# access1 → leaf1 → (10.1.20.0, 10.1.20.1)
# access1 → leaf2 → (10.1.20.2, 10.1.20.3)
```

#### MLAG Peer Links
```python
def allocate_mlag_peer_ips(leaf_pair_num, dc_id):
    # Each pair gets /30 (4 IPs, use 2)
    offset = (leaf_pair_num - 1) * 4 + 1
    odd_leaf_ip = f"10.{dc_id}.255.{offset}"    # .1, .5, .9
    even_leaf_ip = f"10.{dc_id}.255.{offset + 1}" # .2, .6, .10
    return (odd_leaf_ip, even_leaf_ip)

# Example:
# Pair 1 (leaf1-2) → (10.1.255.1, 10.1.255.2)
# Pair 2 (leaf3-4) → (10.1.255.5, 10.1.255.6)
```

### 6. BGP ASN Allocation Logic

```python
def allocate_asn(device_role, pair_number, dc_id, base_asn=65000):
    if device_role == 'spine':
        return base_asn + (dc_id * 100)
    elif device_role == 'leaf':
        return base_asn + (dc_id * 100) + pair_number
    elif device_role == 'borderleaf':
        # Border leafs get ASN +3 from base
        return base_asn + (dc_id * 100) + 3
    else:
        return None  # Access switches don't run BGP

# Examples:
# DC1 Spines → 65100
# DC1 Leaf Pair 1 (leaf1-2) → 65101
# DC1 Leaf Pair 2 (leaf3-4) → 65102
# DC1 Border Leafs → 65103
# DC2 Spines → 65200
```

---

## 📊 Object Relationships & Dependencies

### Dependency Graph
```
Organization
  └── Site
       ├── IP Space Allocation
       └── Datacenter (GENERATOR)
            ├── Computes: leaf_pair_count from number_of_bays
            ├── Generates Devices:
            │    ├── Spines (fixed count)
            │    ├── Leafs (computed count)
            │    ├── Border Leafs (fixed count)
            │    └── Access Switches (= number_of_bays)
            ├── Generates IP Subnets:
            │    ├── Loopback0 subnet
            │    ├── Loopback1 subnet
            │    ├── Spine-Leaf P2P subnet
            │    ├── Leaf-Access P2P subnet
            │    └── MLAG Peer subnet
            ├── For Each Device:
            │    ├── Allocates Management IP
            │    ├── Allocates Loopback IPs (if spine/leaf)
            │    ├── Creates Interfaces (based on role)
            │    ├── Allocates P2P IPs (for each link)
            │    └── Generates BGP Config (if spine/leaf)
            └── Creates MLAG Pairs:
                 └── Assigns peer IPs
```

### Object Creation Order (to respect dependencies)
```
1. Site
2. IP Prefixes (parent subnets)
3. Datacenter (generator starts here)
   ├── 4. Generate Subnets (children of parent subnet)
   ├── 5. Generate Devices (all roles)
   ├── 6. Generate MLAG Pairs (relationships)
   ├── 7. Generate Interfaces (for each device)
   ├── 8. Allocate IP Addresses (for each interface)
   ├── 9. Generate BGP ASNs
   └── 10. Generate BGP Sessions (relationships)
```

---

## 🔄 Scaling Operations

### Adding a New Bay

**Input**: 
- Datacenter: DC1
- New bay_id: 3

**Generator Actions**:
1. Create new access switch: `access3-DC1`
2. Determine leaf pair assignment: 
   - bay_id=3 → pair_index = (3-1)//2 = 1 → Leaf Pair 2 (leaf3-4)
3. Create interfaces:
   - access3-DC1: eth1, eth2 (uplinks)
   - leaf3-DC1: add port for access3
   - leaf4-DC1: add port for access3
4. Allocate IPs:
   - Management: 10.255.0.73
   - P2P to leaf3: (10.1.20.8, 10.1.20.9)
   - P2P to leaf4: (10.1.20.10, 10.1.20.11)
5. Update datacenter: `number_of_bays: 3`

**Result**: New bay operational with zero manual config!

### Adding a New Leaf Pair

**Trigger**: `number_of_bays > (leaf_pair_count * 2)`

**Example**: 
- Current: 2 leaf pairs support 4 bays
- Request: bay_id = 5
- Action: Generate leaf5-DC1 and leaf6-DC1

**Generator Actions**:
1. Create devices: leaf5-DC1, leaf6-DC1
2. Create MLAG pair: [leaf5, leaf6]
3. Allocate IPs:
   - Loopback0: 10.1.0.25, 10.1.0.26
   - Loopback1: 10.1.1.25 (shared)
   - MLAG Peer: 10.1.255.13, 10.1.255.14
4. Create spine uplinks (3 per leaf)
5. Allocate BGP ASN: 65104
6. Generate BGP config

---

## 🎯 Validation Rules

### Pre-Generation Checks
```python
def validate_datacenter_input(dc):
    checks = []
    
    # Check 1: DC ID must be unique
    if dc.dc_id in existing_dc_ids:
        checks.append("ERROR: DC ID already exists")
    
    # Check 2: Parent subnet must be large enough
    required_size = calculate_ip_requirements(dc.number_of_bays)
    if dc.parent_subnet.size < required_size:
        checks.append("ERROR: Parent subnet too small")
    
    # Check 3: Number of bays must be positive
    if dc.number_of_bays < 1:
        checks.append("ERROR: Must have at least 1 bay")
    
    # Check 4: ASN must not conflict
    if asn_conflicts_exist(dc):
        checks.append("ERROR: ASN conflicts detected")
    
    return checks
```

### Post-Generation Checks
```python
def validate_generated_objects(dc):
    checks = []
    
    # Check 1: All devices created
    expected_devices = (
        dc.spine_count + 
        dc.leaf_pair_count * 2 + 
        dc.border_leaf_count + 
        dc.number_of_bays
    )
    if len(dc.devices) != expected_devices:
        checks.append("ERROR: Device count mismatch")
    
    # Check 2: No duplicate IPs
    if has_duplicate_ips(dc.devices):
        checks.append("ERROR: Duplicate IP addresses")
    
    # Check 3: All BGP sessions configured
    if missing_bgp_sessions(dc.devices):
        checks.append("ERROR: Missing BGP sessions")
    
    # Check 4: MLAG pairs valid
    if invalid_mlag_pairs(dc.devices):
        checks.append("ERROR: Invalid MLAG configuration")
    
    return checks
```

---

## 📐 Mermaid Diagram Structure

### High-Level Datacenter Diagram
```mermaid
graph TB
    Site[Site: Paris-DC1]
    DC[Datacenter: DC1<br/>bays=2, spines=3]
    
    Site --> DC
    
    DC --> Spines[Spine Layer<br/>spine1-3-DC1]
    DC --> Leafs[Leaf Layer<br/>leaf1-4-DC1]
    DC --> Borders[Border Layer<br/>borderleaf1-2-DC1]
    DC --> Bays[Bay Layer<br/>bay1-2]
    
    Bays --> Bay1[Bay 1]
    Bays --> Bay2[Bay 2]
    
    Bay1 --> Access1[access1-DC1]
    Bay2 --> Access2[access2-DC1]
    
    Access1 --> LeafPair1[Leaf Pair 1<br/>leaf1-2-DC1]
    Access2 --> LeafPair2[Leaf Pair 2<br/>leaf3-4-DC1]
    
    LeafPair1 --> Spines
    LeafPair2 --> Spines
    Borders --> Spines
```

### Detailed Device-Level Diagram
```mermaid
graph LR
    subgraph DC1
        S1[spine1-DC1<br/>10.1.0.11]
        S2[spine2-DC1<br/>10.1.0.12]
        S3[spine3-DC1<br/>10.1.0.13]
        
        L1[leaf1-DC1<br/>10.1.0.21<br/>VTEP: 10.1.1.21]
        L2[leaf2-DC1<br/>10.1.0.22<br/>VTEP: 10.1.1.21]
        L3[leaf3-DC1<br/>10.1.0.23<br/>VTEP: 10.1.1.23]
        L4[leaf4-DC1<br/>10.1.0.24<br/>VTEP: 10.1.1.23]
        
        A1[access1-DC1<br/>Bay 1]
        A2[access2-DC1<br/>Bay 2]
        
        L1 -.MLAG.- L2
        L3 -.MLAG.- L4
        
        S1 --> L1
        S1 --> L2
        S1 --> L3
        S1 --> L4
        
        S2 --> L1
        S2 --> L2
        S2 --> L3
        S2 --> L4
        
        S3 --> L1
        S3 --> L2
        S3 --> L3
        S3 --> L4
        
        L1 --> A1
        L2 --> A1
        L3 --> A2
        L4 --> A2
    end
```

---

## 🚀 Summary: What Infrahub Will Generate

**From Single "Datacenter" Object Input**:

✅ **27 Device Objects** (for DC with 2 bays):
- 3 Spines
- 4 Leafs
- 2 Border Leafs
- 2 Access switches

✅ **~150 Interface Objects**:
- Physical interfaces
- Loopback interfaces
- VLAN interfaces (MLAG)

✅ **~90 IP Address Objects**:
- Management IPs
- Loopback IPs
- P2P link IPs
- MLAG peer IPs

✅ **~40 BGP Session Objects**:
- Spine-to-Leaf sessions
- Leaf MLAG iBGP sessions
- Border-to-DCI sessions

✅ **3 MLAG Pair Objects**:
- Leaf pairs
- Border leaf pair

✅ **6 Subnet Objects**:
- Loopback0, Loopback1
- Spine-Leaf P2P, Leaf-Access P2P
- MLAG Peer, Tenant VLANs

**All from ~10 input attributes!** 🎯

This is the power of the generator pattern - infrastructure as data!