docs: Add comprehensive README with architecture and project phases

2025-12-20 15:36:14 +00:00
parent cac41f330d
commit 9102bc1318
1 changed files with 216 additions and 2 deletions
--- a/README.md
+++ b/README.md
@@ -1,3 +1,217 @@
-# fabric-orchestrator
+# Fabric Orchestrator

-Declarative Network Fabric Orchestrator - Terraform-like infrastructure management for Arista EVPN-VXLAN using gNMI, YANG, and NetBox as Source of Truth
+**Declarative Network Infrastructure Management for Arista EVPN-VXLAN Fabrics**
+
+A Terraform-like orchestration system that uses NetBox as Source of Truth and gNMI/YANG for atomic configuration management of Arista data center fabrics.
+
+## 🎯 Project Vision
+
+Transform network infrastructure management from imperative scripting to true declarative infrastructure-as-code, where:
+
+- **Intent** is defined in NetBox (ConfigContexts, Custom Fields)
+- **State** is continuously monitored via gNMI Subscribe
+- **Changes** are computed as diffs and applied atomically via gNMI Set
+- **Drift** is detected and optionally auto-remediated
+
+Think `terraform plan` and `terraform apply`, but for your network fabric.
+
+## 🏗️ Architecture
+
+```
+┌─────────────────────────────────────────────────────────────────────────────┐
+│                           INTENT LAYER                                       │
+│  ┌─────────────┐    ┌──────────────────┐    ┌─────────────────────────────┐ │
+│  │   NetBox    │    │  ConfigContexts  │    │  Custom Fields / Tags       │ │
+│  │  (SoT)      │◄──►│  (Structured     │◄──►│  (VLAN, VNI, VRF, BGP AS)   │ │
+│  │             │    │   Intent Data)   │    │                             │ │
+│  └──────┬──────┘    └──────────────────┘    └─────────────────────────────┘ │
+└─────────┼───────────────────────────────────────────────────────────────────┘
+          │ Webhook / Polling
+          ▼
+┌─────────────────────────────────────────────────────────────────────────────┐
+│                        ORCHESTRATION LAYER                                   │
+│  ┌─────────────────────────────────────────────────────────────────────────┐│
+│  │                     State Reconciliation Engine                         ││
+│  │  ┌───────────────┐   ┌───────────────┐   ┌───────────────────────────┐ ││
+│  │  │ Intent Parser │   │  Diff Engine  │   │  Transaction Planner      │ ││
+│  │  │ (NetBox→YANG) │──►│ (Want vs Have)│──►│  (Ordered gNMI SetReqs)   │ ││
+│  │  └───────────────┘   └───────────────┘   └───────────────────────────┘ ││
+│  └─────────────────────────────────────────────────────────────────────────┘│
+│                                    │                                         │
+│  ┌─────────────────────────────────┼───────────────────────────────────────┐│
+│  │              Event Bus (Redis / NATS)                                   ││
+│  │   • config_drift_detected    • intent_changed    • apply_complete      ││
+│  └─────────────────────────────────────────────────────────────────────────┘│
+└─────────────────────────────────────────────────────────────────────────────┘
+          │ gNMI Subscribe (Telemetry)          │ gNMI Set (Config)
+          ▼                                     ▼
+┌─────────────────────────────────────────────────────────────────────────────┐
+│                         DEVICE LAYER                                         │
+│  ┌────────────┐ ┌────────────┐ ┌────────────┐ ┌────────────┐               │
+│  │  spine1    │ │  spine2    │ │  leaf1     │ │  leaf2     │  ...          │
+│  │  gNMI:6030 │ │  gNMI:6030 │ │  gNMI:6030 │ │  gNMI:6030 │               │
+│  └────────────┘ └────────────┘ └────────────┘ └────────────┘               │
+└─────────────────────────────────────────────────────────────────────────────┘
+```
+
+## 🔧 Target Fabric
+
+This project is designed for the Arista EVPN-VXLAN ContainerLab topology:
+
+- **2 Spines** (BGP Route Reflectors, AS 65000)
+- **8 Leafs** (4 MLAG VTEP pairs, AS 65001-65004)
+- **cEOS 4.35.0F** with gNMI enabled
+- **EVPN Type-2** (L2 VXLAN) and **Type-5** (L3 VXLAN) support
+
+Reference: [arista-evpn-vxlan-clab](https://gitea.arnodo.fr/Damien/arista-evpn-vxlan-clab)
+
+## 📋 Project Phases
+
+### Phase 1: YANG Path Discovery (Weeks 1-2)
+> **Goal**: Map Arista EOS 4.35.0F YANG models and validate gNMI connectivity
+
+- [ ] Enable and test gNMI on cEOS devices
+- [ ] Explore available YANG models (OpenConfig vs Native Arista)
+- [ ] Document working paths for: VLANs, Interfaces, BGP, VXLAN, MLAG
+- [ ] Create discovery tooling for path exploration
+- [ ] Build YANG path reference documentation
+
+### Phase 2: Minimal Reconciler (Weeks 3-4)
+> **Goal**: Prove the reconciliation pattern with VLANs and VNI mappings
+
+- [ ] Define NetBox ConfigContext schema for intent
+- [ ] Implement NetBox → YANG path mapper (VLANs only)
+- [ ] Build basic diff engine (current vs desired state)
+- [ ] Implement gNMI Get for state retrieval
+- [ ] Implement gNMI Set for atomic changes
+- [ ] Create CLI with `plan` and `apply` commands
+
+### Phase 3: Full Fabric Coverage (Weeks 5-8)
+> **Goal**: Extend to complete EVPN-VXLAN fabric configuration
+
+- [ ] Add BGP underlay configuration
+- [ ] Add BGP EVPN overlay configuration
+- [ ] Add MLAG configuration (with coordinated peer updates)
+- [ ] Add VRF and L3 VXLAN support
+- [ ] Add interface configuration
+- [ ] Implement dependency ordering for changes
+- [ ] Add rollback capability
+
+### Phase 4: Event-Driven Automation (Weeks 9-10)
+> **Goal**: Real-time drift detection and NetBox webhook integration
+
+- [ ] Implement gNMI Subscribe for config change telemetry
+- [ ] Build drift detection engine
+- [ ] Add NetBox webhook receiver (FastAPI)
+- [ ] Implement event bus (Redis/NATS)
+- [ ] Add auto-remediation option
+- [ ] Create operational dashboard/logging
+
+## 📁 Project Structure
+
+```
+fabric-orchestrator/
+├── README.md
+├── pyproject.toml
+├── docker-compose.yml              # Redis, API server
+├── src/
+│   ├── __init__.py
+│   ├── cli.py                      # CLI interface (plan, apply, drift)
+│   ├── api.py                      # FastAPI server for webhooks
+│   ├── reconciler/
+│   │   ├── __init__.py
+│   │   ├── engine.py               # Core reconciliation logic
+│   │   ├── diff.py                 # State comparison
+│   │   └── planner.py              # Change ordering/dependencies
+│   ├── yang/
+│   │   ├── __init__.py
+│   │   ├── mapper.py               # NetBox intent → YANG paths
+│   │   ├── paths.py                # YANG path definitions
+│   │   └── validators.py           # Schema validation
+│   ├── gnmi/
+│   │   ├── __init__.py
+│   │   ├── client.py               # gNMI client wrapper
+│   │   └── transactions.py         # Atomic operations
+│   ├── netbox/
+│   │   ├── __init__.py
+│   │   ├── client.py               # NetBox API client
+│   │   └── models.py               # Intent data models
+│   └── events/
+│       ├── __init__.py
+│       ├── handlers.py             # Event handlers
+│       └── bus.py                  # Event bus (Redis)
+├── tests/
+│   ├── test_reconciler.py
+│   ├── test_yang_mapper.py
+│   └── fixtures/
+│       ├── netbox_intent.json
+│       └── device_state.json
+└── docs/
+    ├── architecture.md
+    ├── yang-paths.md               # Documented YANG paths
+    └── netbox-schema.md            # ConfigContext schema
+```
+
+## 🛠️ Technology Stack
+
+| Component | Technology | Purpose |
+|-----------|------------|---------|
+| Source of Truth | NetBox | Intent definition via ConfigContexts |
+| Transport | gNMI | Configuration and telemetry |
+| Data Models | YANG (OpenConfig + Arista) | Structured configuration |
+| Orchestrator | Python (asyncio) | Reconciliation engine |
+| CLI | Click + Rich | User interface |
+| API | FastAPI | Webhook receiver |
+| Event Bus | Redis | Async event handling |
+| Lab | ContainerLab + cEOS | Development environment |
+
+## 🔗 Related Projects
+
+- [arista-evpn-vxlan-clab](https://gitea.arnodo.fr/Damien/arista-evpn-vxlan-clab) - Target fabric topology
+- [projet-vxlan-automation](https://gitea.arnodo.fr/Damien/projet-vxlan-automation) - Previous NetBox RenderConfig work
+- [Arista YANG Models](https://github.com/aristanetworks/yang/tree/master/EOS-4.35.0F) - EOS 4.35.0F YANG definitions
+
+## 📚 References
+
+### YANG / gNMI
+- [Arista gNMI Documentation](https://aristanetworks.github.io/openmgmt/configuration/gnmi/)
+- [OpenConfig Models](https://github.com/openconfig/public)
+- [pygnmi Library](https://github.com/akarneliuk/pygnmi)
+
+### EVPN-VXLAN
+- [Arista BGP EVPN Configuration Example](https://overlaid.net/2019/01/27/arista-bgp-evpn-configuration-example/)
+- [Arista EVPN Deployment Guide](https://www.arista.com/en/solutions/evpn-vxlan)
+
+### Event-Driven Automation
+- [NetBox Webhooks](https://docs.netbox.dev/en/stable/integrations/webhooks/)
+- [Event-Driven Ansible](https://www.ansible.com/blog/introducing-event-driven-ansible)
+
+## 🚀 Getting Started
+
+*Coming in Phase 1*
+
+```bash
+# Clone the repository
+git clone https://gitea.arnodo.fr/Damien/fabric-orchestrator.git
+cd fabric-orchestrator
+
+# Install dependencies
+pip install -e .
+
+# Verify gNMI connectivity to your fabric
+fabric-orch discover --target leaf1:6030
+
+# Generate execution plan
+fabric-orch plan
+
+# Apply changes
+fabric-orch apply
+```
+
+## 📄 License
+
+MIT License - See [LICENSE](LICENSE) for details.
+
+---
+
+**Status**: 🚧 Active Development - Phase 1