Damien/infra-clab-pipeline
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
0232d46b4a117214b04a73ebfcb706dfbf83a5df
infra-clab-pipeline/infrahub/generator.md
darnodo d93a9e5c29 Add network schema for devices, interfaces, VLANs, VRFs, BGP configurations, MLAG, EVPN, VXLAN, routing policies, and DCI connections 
- Created device schema with attributes for hostname, role, platform, and management IP.
- Added interface schema with various types, MTU, speed, and switchport modes.
- Introduced VLAN and VRF schemas with relationships to devices and interfaces.
- Implemented BGP configuration, peer groups, and neighbors with detailed attributes.
- Established MLAG domain and interface schemas for multi-chassis link aggregation.
- Developed EVPN and VXLAN schemas for overlay networking.
- Created routing policy schemas including route maps and prefix lists.
- Added DCI switch and connection schemas for inter-datacenter connectivity.
2025-11-12 11:32:26 +01:00

14 KiB
Raw Blame History

Datacenter Generator - Visual Diagrams

1. Generator Concept - High Level

graph TB
    subgraph Input["📥 INPUT"]
        DC1[Datacenter: DC1<br/>---<br/>dc_id: 1<br/>bays: 2<br/>dci_enabled: false]
        DC2[Datacenter: DC2<br/>---<br/>dc_id: 2<br/>bays: 2<br/>dci_enabled: false]
        DCI_Config[DCI Configuration<br/>---<br/>Manual/External<br/>IP: 10.253.254.x]
    end
    
    subgraph Logic["⚙️ DATACENTER GENERATOR"]
        Gen1[DC1 Generator:<br/>Create 11 devices<br/>Border eth12: shutdown]
        Gen2[DC2 Generator:<br/>Create 11 devices<br/>Border eth12: shutdown]
    end
    
    subgraph Output1["📤 DC1 OUTPUT"]
        Dev1[11 Device Objects]
        Int1[~50 Interface Objects]
        IP1[~30 IP Addresses]
    end
    
    subgraph Output2["📤 DC2 OUTPUT"]
        Dev2[11 Device Objects]
        Int2[~50 Interface Objects]
        IP2[~30 IP Addresses]
    end
    
    subgraph Manual["🔧 MANUAL DCI SETUP"]
        DCIDevice[DCI Device<br/>Loopback: 10.253.0.1<br/>ASN: 65000]
        UpdateDC1[Update DC1:<br/>dci_enabled: true<br/>dci_remote_dc_id: 2]
        UpdateDC2[Update DC2:<br/>dci_enabled: true<br/>dci_remote_dc_id: 1]
    end
    
    DC1 --> Gen1
    DC2 --> Gen2
    
    Gen1 --> Dev1
    Gen1 --> Int1
    Gen1 --> IP1
    
    Gen2 --> Dev2
    Gen2 --> Int2
    Gen2 --> IP2
    
    Output1 --> Manual
    Output2 --> Manual
    DCI_Config --> DCIDevice
    DCIDevice --> UpdateDC1
    DCIDevice --> UpdateDC2
    
    style DC1 fill:#e1f5ff
    style DC2 fill:#e1f5ff
    style Manual fill:#fff9c4
    style DCIDevice fill:#ffccbc

2. Datacenter Hierarchy

graph TB
    Org[Organization]
    Site[Site: Paris-DC1<br/>Location: Paris, France]
    DC[Datacenter: DC1<br/>dc_id: 1<br/>bays: 2<br/>spines: 3]
    
    Org --> Site
    Site --> DC
    
    DC --> SpineLayer[Spine Layer]
    DC --> LeafLayer[Leaf Layer]
    DC --> BorderLayer[Border Layer]
    DC --> BayLayer[Bay Layer]
    DC --> Subnets[IP Subnets]
    
    SpineLayer --> S1[spine1-DC1<br/>ASN: 65100]
    SpineLayer --> S2[spine2-DC1<br/>ASN: 65100]
    SpineLayer --> S3[spine3-DC1<br/>ASN: 65100]
    
    LeafLayer --> LP1[Leaf Pair 1<br/>ASN: 65101]
    LeafLayer --> LP2[Leaf Pair 2<br/>ASN: 65102]
    
    LP1 --> L1[leaf1-DC1<br/>10.1.0.21]
    LP1 --> L2[leaf2-DC1<br/>10.1.0.22]
    LP2 --> L3[leaf3-DC1<br/>10.1.0.23]
    LP2 --> L4[leaf4-DC1<br/>10.1.0.24]
    
    BorderLayer --> BP[Border Pair<br/>ASN: 65103]
    BP --> B1[borderleaf1-DC1]
    BP --> B2[borderleaf2-DC1]
    
    BayLayer --> Bay1[Bay 1]
    BayLayer --> Bay2[Bay 2]
    
    Bay1 --> A1[access1-DC1]
    Bay2 --> A2[access2-DC1]
    
    Subnets --> Sub1[10.1.0.0/24<br/>Loopback0]
    Subnets --> Sub2[10.1.1.0/24<br/>Loopback1]
    Subnets --> Sub3[10.1.10.0/24<br/>Spine-Leaf P2P]
    Subnets --> Sub4[10.1.20.0/24<br/>Leaf-Access P2P]
    
    style DC fill:#ffecb3
    style LP1 fill:#b3e5fc
    style LP2 fill:#b3e5fc
    style Bay1 fill:#c8e6c9
    style Bay2 fill:#c8e6c9

3. Bay-to-Leaf Assignment Logic

graph LR
    subgraph Bays["🏢 Bays"]
        Bay1[Bay 1<br/>access1-DC1]
        Bay2[Bay 2<br/>access2-DC1]
        Bay3[Bay 3<br/>access3-DC1]
        Bay4[Bay 4<br/>access4-DC1]
    end
    
    subgraph LeafPairs["🔀 Leaf Pairs - MLAG"]
        LP1[Leaf Pair 1<br/>leaf1-DC1 ↔ leaf2-DC1<br/>ASN: 65101]
        LP2[Leaf Pair 2<br/>leaf3-DC1 ↔ leaf4-DC1<br/>ASN: 65102]
    end
    
    subgraph Spines["⬆️ Spine Layer"]
        S1[spine1-DC1]
        S2[spine2-DC1]
        S3[spine3-DC1]
    end
    
    Bay1 -.2 uplinks.- LP1
    Bay2 -.2 uplinks.- LP1
    Bay3 -.2 uplinks.- LP2
    Bay4 -.2 uplinks.- LP2
    
    LP1 --> S1
    LP1 --> S2
    LP1 --> S3
    LP2 --> S1
    LP2 --> S2
    LP2 --> S3
    
    style Bay1 fill:#c8e6c9
    style Bay2 fill:#c8e6c9
    style Bay3 fill:#ffccbc
    style Bay4 fill:#ffccbc
    style LP1 fill:#b3e5fc
    style LP2 fill:#b3e5fc

4. Complete DC1 Physical Topology

graph TB
    subgraph DCI["DCI LAYER - Inter-DC"]
        DCID[DCI Switch<br/>10.253.0.1<br/>ASN: 65000]
    end
    
    subgraph Border["BORDER LEAF - DCI Gateway"]
        B1[borderleaf1-DC1<br/>10.1.0.31<br/>eth12: shutdown or active]
        B2[borderleaf2-DC1<br/>10.1.0.32<br/>eth12: shutdown or active]
    end
    
    subgraph Spine["SPINE LAYER - L3 Core"]
        S1[spine1-DC1<br/>10.1.0.11<br/>ASN: 65100]
        S2[spine2-DC1<br/>10.1.0.12<br/>ASN: 65100]
        S3[spine3-DC1<br/>10.1.0.13<br/>ASN: 65100]
    end
    
    subgraph Leaf["LEAF LAYER - Aggregation + VXLAN"]
        subgraph Pair1["MLAG Pair 1 - ASN: 65101"]
            L1[leaf1-DC1<br/>10.1.0.21<br/>VTEP: 10.1.1.21]
            L2[leaf2-DC1<br/>10.1.0.22<br/>VTEP: 10.1.1.21]
        end
        subgraph Pair2["MLAG Pair 2 - ASN: 65102"]
            L3[leaf3-DC1<br/>10.1.0.23<br/>VTEP: 10.1.1.23]
            L4[leaf4-DC1<br/>10.1.0.24<br/>VTEP: 10.1.1.23]
        end
    end
    
    subgraph Access["ACCESS LAYER - Rack/Bay ToR"]
        A1[access1-DC1<br/>Bay 1<br/>L2 Switch]
        A2[access2-DC1<br/>Bay 2<br/>L2 Switch]
    end
    
    subgraph Hosts["HOST LAYER"]
        H1[host1-DC1<br/>172.16.100.10]
        H2[host2-DC1<br/>172.16.200.10]
    end
    
    DCID -.Optional DCI.- B1
    DCID -.Optional DCI.- B2
    
    S1 -.eBGP.- L1
    S1 -.eBGP.- L2
    S1 -.eBGP.- L3
    S1 -.eBGP.- L4
    S1 -.eBGP.- B1
    S1 -.eBGP.- B2
    
    S2 -.eBGP.- L1
    S2 -.eBGP.- L2
    S2 -.eBGP.- L3
    S2 -.eBGP.- L4
    S2 -.eBGP.- B1
    S2 -.eBGP.- B2
    
    S3 -.eBGP.- L1
    S3 -.eBGP.- L2
    S3 -.eBGP.- L3
    S3 -.eBGP.- L4
    S3 -.eBGP.- B1
    S3 -.eBGP.- B2
    
    L1 ---|MLAG| L2
    L3 ---|MLAG| L4
    B1 ---|MLAG| B2
    
    L1 -->|eth7| A1
    L2 -->|eth7| A1
    L3 -->|eth7| A2
    L4 -->|eth7| A2
    
    A1 -->|eth10| H1
    A2 -->|eth10| H2
    
    style DCID fill:#ffccbc
    style S1 fill:#ffccbc
    style S2 fill:#ffccbc
    style S3 fill:#ffccbc
    style L1 fill:#b3e5fc
    style L2 fill:#b3e5fc
    style L3 fill:#b3e5fc
    style L4 fill:#b3e5fc
    style A1 fill:#c8e6c9
    style A2 fill:#c8e6c9
    style B1 fill:#f8bbd0
    style B2 fill:#f8bbd0

5. IP Address Generation Flow

graph TB
    Start[Datacenter Object<br/>dc_id: 1, bays: 2]
    
    Start --> GenSubnets[Generate Subnets<br/>from dc_id]
    
    GenSubnets --> Sub1[10.1.0.0/24<br/>Loopback0]
    GenSubnets --> Sub2[10.1.1.0/24<br/>Loopback1]
    GenSubnets --> Sub3[10.1.10.0/24<br/>Spine-Leaf P2P]
    GenSubnets --> Sub4[10.1.20.0/24<br/>Leaf-Access P2P]
    GenSubnets --> Sub5[10.1.255.0/24<br/>MLAG Peer]
    GenSubnets --> Sub6[10.255.0.0/24<br/>Management]
    
    Sub1 --> AllocLo0[Allocate Loopback0<br/>to Spines & Leafs]
    Sub2 --> AllocLo1[Allocate Loopback1<br/>to Leafs - shared in pairs]
    Sub3 --> AllocP2P1[Allocate /31s<br/>for Spine-Leaf links]
    Sub4 --> AllocP2P2[Allocate /31s<br/>for Leaf-Access links]
    Sub5 --> AllocMLAG[Allocate /30s<br/>for MLAG peers]
    Sub6 --> AllocMgmt[Allocate Management IPs<br/>to all devices]
    
    AllocLo0 --> Spine1IP[spine1: 10.1.0.11/32]
    AllocLo0 --> Leaf1IP[leaf1: 10.1.0.21/32]
    
    AllocLo1 --> Leaf1VTEP[leaf1-2: 10.1.1.21/32 - shared]
    
    AllocP2P1 --> Link1[spine1-leaf1: 10.1.10.0/31]
    
    AllocP2P2 --> Link2[leaf1-access1: 10.1.20.0/31]
    
    AllocMLAG --> MLAG1[leaf1-2 peer: 10.1.255.1-2/30]
    
    AllocMgmt --> Mgmt1[spine1: 10.255.0.11]
    
    style Start fill:#ffecb3
    style Sub1 fill:#e1f5ff
    style Sub2 fill:#e1f5ff
    style Sub3 fill:#e1f5ff
    style Sub4 fill:#e1f5ff
    style Sub5 fill:#e1f5ff
    style Sub6 fill:#e1f5ff

6. Device Generation Flow

graph TB
    DC[Datacenter: DC1<br/>bays: 2]
    
    DC --> CalcLeafs[Calculate:<br/>leaf_pairs = ceil - bays / 2 - = 1<br/>total_leafs = 1 * 2 = 2]
    
    CalcLeafs --> GenSpines[Generate Spines<br/>count: 3 - fixed]
    CalcLeafs --> GenLeafs[Generate Leafs<br/>count: 4 - computed]
    CalcLeafs --> GenBorders[Generate Border Leafs<br/>count: 2 - fixed]
    CalcLeafs --> GenAccess[Generate Access<br/>count: 2 - from bays]
    
    GenSpines --> S1[spine1-DC1]
    GenSpines --> S2[spine2-DC1]
    GenSpines --> S3[spine3-DC1]
    
    GenLeafs --> LP1[Create MLAG Pair 1]
    GenLeafs --> LP2[Create MLAG Pair 2]
    
    LP1 --> L1[leaf1-DC1]
    LP1 --> L2[leaf2-DC1]
    LP2 --> L3[leaf3-DC1]
    LP2 --> L4[leaf4-DC1]
    
    GenBorders --> BP[Create Border Pair]
    BP --> B1[borderleaf1-DC1]
    BP --> B2[borderleaf2-DC1]
    
    GenAccess --> AssignBays[Assign Bays to Leaf Pairs]
    AssignBays --> A1Config[Bay 1 → Leaf Pair 1<br/>access1-DC1]
    AssignBays --> A2Config[Bay 2 → Leaf Pair 1<br/>access2-DC1]
    
    A1Config --> A1[access1-DC1]
    A2Config --> A2[access2-DC1]
    
    style DC fill:#ffecb3
    style LP1 fill:#b3e5fc
    style LP2 fill:#b3e5fc
    style BP fill:#f8bbd0

7. Scaling Scenario - Adding Bay 3

graph LR
    subgraph Current["📦 Current State - 2 Bays"]
        CurDC[Datacenter: DC1<br/>bays: 2<br/>leaf_pairs: 1]
        CurLP1[Leaf Pair 1<br/>leaf1-2]
        CurBay1[Bay 1 → access1]
        CurBay2[Bay 2 → access2]
        
        CurDC --> CurLP1
        CurLP1 --> CurBay1
        CurLP1 --> CurBay2
    end
    
    subgraph Action["⚙️ User Action"]
        AddBay[Add Bay 3<br/>number_of_bays: 2 → 3]
    end
    
    subgraph Generator["🔄 Generator Logic"]
        Check[Check:<br/>bays=3 > pairs*2=2?<br/>YES → Need new pair]
        CreatePair[Create Leaf Pair 2<br/>leaf3-DC1, leaf4-DC1]
        CreateAccess[Create access3-DC1]
        AssignBay[Assign Bay 3 → Pair 2]
        AllocIPs[Allocate IPs]
        CreateLinks[Create Interfaces]
        ConfigBGP[Configure BGP<br/>ASN: 65102]
    end
    
    subgraph Result["✅ New State - 3 Bays"]
        NewDC[Datacenter: DC1<br/>bays: 3<br/>leaf_pairs: 2]
        NewLP1[Leaf Pair 1<br/>leaf1-2<br/>ASN: 65101]
        NewLP2[Leaf Pair 2<br/>leaf3-4<br/>ASN: 65102]
        NewBay1[Bay 1 → access1]
        NewBay2[Bay 2 → access2]
        NewBay3[Bay 3 → access3]
        
        NewDC --> NewLP1
        NewDC --> NewLP2
        NewLP1 --> NewBay1
        NewLP1 --> NewBay2
        NewLP2 --> NewBay3
    end
    
    Current --> AddBay
    AddBay --> Check
    Check --> CreatePair
    CreatePair --> CreateAccess
    CreateAccess --> AssignBay
    AssignBay --> AllocIPs
    AllocIPs --> CreateLinks
    CreateLinks --> ConfigBGP
    ConfigBGP --> Result
    
    style AddBay fill:#fff59d
    style Check fill:#ffccbc
    style Result fill:#c8e6c9

8. Infrahub Generator Workflow

sequenceDiagram
    participant User
    participant Infrahub
    participant Generator
    participant Validator
    participant GraphQL
    participant Devices
    
    User->>Infrahub: Create Datacenter Object<br/>(name, dc_id, bays, etc.)
    Infrahub->>Validator: Pre-Generation Validation
    Validator-->>Infrahub: ✅ Valid Input
    
    Infrahub->>Generator: Trigger Generator
    
    Generator->>Generator: Compute Derived Values<br/>(leaf_pairs, subnets, etc.)
    
    loop For Each Layer
        Generator->>Devices: Create Spine Devices
        Generator->>Devices: Create Leaf Devices
        Generator->>Devices: Create Access Devices
    end
    
    loop For Each Device
        Generator->>Devices: Create Interfaces
        Generator->>Devices: Allocate IP Addresses
        Generator->>Devices: Generate BGP Config
    end
    
    Generator->>Validator: Post-Generation Validation
    Validator-->>Generator: ✅ All Objects Valid
    
    Generator->>GraphQL: Commit All Objects
    GraphQL-->>Infrahub: Objects Created
    
    Infrahub-->>User: ✅ Datacenter Generated<br/>27 devices, 150 interfaces, 90 IPs

9. Configuration Generation Flow

graph TB
    subgraph Infrahub["📊 Infrahub - Source of Truth"]
        DeviceDB[(Device Objects<br/>Interfaces<br/>IPs<br/>BGP Sessions)]
    end
    
    subgraph Generator["⚙️ Config Generator"]
        Templates[Jinja2 Templates<br/>by Device Role]
        RenderEngine[Template Renderer]
    end
    
    subgraph Output["📄 Generated Configs"]
        SpineConfig[spine1-DC1.cfg]
        LeafConfig[leaf1-DC1.cfg]
        AccessConfig[access1-DC1.cfg]
    end
    
    subgraph Deployment["🚀 Deployment"]
        Validation[Config Validation]
        Push[Push to Devices<br/>via eAPI/NETCONF]
    end
    
    DeviceDB -->|Query Device Data| RenderEngine
    Templates --> RenderEngine
    
    RenderEngine --> SpineConfig
    RenderEngine --> LeafConfig
    RenderEngine --> AccessConfig
    
    SpineConfig --> Validation
    LeafConfig --> Validation
    AccessConfig --> Validation
    
    Validation --> Push
    
    Push --> Devices[Physical Devices]
    
    style Infrahub fill:#e1f5ff
    style Generator fill:#fff9c4
    style Output fill:#c8e6c9
    style Deployment fill:#ffccbc

10. Complete Data Model Relationships

erDiagram
    ORGANIZATION ||--o{ SITE : contains
    SITE ||--o{ DATACENTER : contains
    SITE ||--o{ IP_PREFIX : manages
    
    DATACENTER ||--|{ SUBNET : generates
    DATACENTER ||--|{ DEVICE : generates
    DATACENTER ||--|{ MLAG_PAIR : generates
    
    DEVICE ||--|{ INTERFACE : has
    INTERFACE ||--o| IP_ADDRESS : assigned
    INTERFACE ||--o| BGP_SESSION : endpoint
    
    DEVICE }|--|| ROLE : has
    DEVICE }o--o| MLAG_PAIR : member_of
    
    MLAG_PAIR ||--|| DEVICE : primary
    MLAG_PAIR ||--|| DEVICE : secondary
    
    BGP_SESSION }o--|| INTERFACE : local
    BGP_SESSION }o--|| INTERFACE : remote
    
    SUBNET ||--|{ IP_ADDRESS : contains
    IP_PREFIX ||--|{ SUBNET : parent
    
    ORGANIZATION {
        string name
    }
    
    SITE {
        string name
        string location
    }
    
    DATACENTER {
        string name
        int dc_id
        int number_of_bays
        int spine_count
        string parent_subnet
    }
    
    DEVICE {
        string hostname
        string role
        string mgmt_ip
    }
    
    INTERFACE {
        string name
        string type
        int mtu
    }
    
    IP_ADDRESS {
        string address
        int prefix_length
    }
    
    BGP_SESSION {
        int local_asn
        int remote_asn
    }
Reference in New Issue View Git Blame Copy Permalink