diff --git a/.gitignore b/.gitignore index 93f4f93..07cadff 100644 --- a/.gitignore +++ b/.gitignore @@ -1,2 +1,3 @@ __pycache__ .envrc +.ruff_cache diff --git a/README.md b/README.md index edd6a2c..0b9af56 100644 --- a/README.md +++ b/README.md @@ -24,13 +24,13 @@ Think `terraform plan` and `terraform apply`, but for your network fabric — po We chose [InfraHub](https://github.com/opsmill/infrahub) over NetBox as Source of Truth for several reasons: -| Feature | NetBox | InfraHub | -| ------------------- | ----------------------- | -------------------------------------------- | -| **Schema** | Fixed DCIM/IPAM model | Fully customizable YAML schema | -| **Git Integration** | External sync needed | Native - branches = data branches | -| **Versioning** | Changelog only | True Git-like versioning with merges | -| **Transforms** | Limited | Built-in Jinja2 + Python transforms | -| **GraphQL** | Yes | Yes (auto-generated from schema) | +| Feature | NetBox | InfraHub | +| ------------------- | --------------------- | ------------------------------------ | +| **Schema** | Fixed DCIM/IPAM model | Fully customizable YAML schema | +| **Git Integration** | External sync needed | Native - branches = data branches | +| **Versioning** | Changelog only | True Git-like versioning with merges | +| **Transforms** | Limited | Built-in Jinja2 + Python transforms | +| **GraphQL** | Yes | Yes (auto-generated from schema) | **Key benefits for this project:** @@ -41,16 +41,16 @@ We chose [InfraHub](https://github.com/opsmill/infrahub) over NetBox as Source o ## 🎛 Why Prefect? -| Feature | Benefit | -| -------------------------------- | ------------------------------------------------------------------------- | -| **Python-native workflows** | Use `@flow` and `@task` decorators — no YAML, just Python | -| **Free secrets management** | Native `Secret` blocks for credentials (free in OSS) | -| **Built-in UI** | Dashboard, logs, metrics, execution history via `prefect server start` | -| **No containerization required** | Run flows directly with `.serve()` — no Docker needed | -| **Event-driven triggers** | Schedule, webhooks (via FastAPI), flow triggers out of the box | -| **Task dependencies** | Automatic dependency ordering via task result passing or `wait_for` | -| **Retry & error handling** | Built-in retry policies with `@task(retries=3)` | -| **Human-in-the-loop** | Native `pause_flow_run()` for approval workflows | +| Feature | Benefit | +| -------------------------------- | ---------------------------------------------------------------------- | +| **Python-native workflows** | Use `@flow` and `@task` decorators — no YAML, just Python | +| **Free secrets management** | Native `Secret` blocks for credentials (free in OSS) | +| **Built-in UI** | Dashboard, logs, metrics, execution history via `prefect server start` | +| **No containerization required** | Run flows directly with `.serve()` — no Docker needed | +| **Event-driven triggers** | Schedule, webhooks (via FastAPI), flow triggers out of the box | +| **Task dependencies** | Automatic dependency ordering via task result passing or `wait_for` | +| **Retry & error handling** | Built-in retry policies with `@task(retries=3)` | +| **Human-in-the-loop** | Native `pause_flow_run()` for approval workflows | ## 🎯 Target Fabric @@ -67,12 +67,12 @@ Reference lab topology: [arista-evpn-vxlan-clab](https://gitea.arnodo.fr/Damien/ Progress is tracked via issues. See [all issues](https://gitea.arnodo.fr/Damien/fabric-orchestrator/issues) or filter by phase: -| Phase | Description | Status | -| ----------- | ---------------------------------------------------------------------------- | -------------- | -| **Phase 1** | YANG Path Discovery - Map EOS 4.35.0F YANG models, validate gNMI | ✅ Complete | -| **Phase 2** | InfraHub Client & Core Reconciler - SDK client, diff engine, YANG mappers | 🔄 In Progress | -| **Phase 3** | Full Fabric Coverage - BGP, MLAG, VRFs mappers | 📋 Planned | -| **Phase 4** | Prefect Integration - Flows, webhooks, drift detection | 📋 Planned | +| Phase | Description | Status | +| ----------- | ------------------------------------------------------------------------- | -------------- | +| **Phase 1** | YANG Path Discovery - Map EOS 4.35.0F YANG models, validate gNMI | ✅ Complete | +| **Phase 2** | InfraHub Client & Core Reconciler - SDK client, diff engine, YANG mappers | 🔄 In Progress | +| **Phase 3** | Full Fabric Coverage - BGP, MLAG, VRFs mappers | 📋 Planned | +| **Phase 4** | Prefect Integration - Flows, webhooks, drift detection | 📋 Planned | ## 📁 Project Structure @@ -115,16 +115,16 @@ fabric-orchestrator/ ## 🛠️ Technology Stack -| Component | Technology | Purpose | -| --------------- | ------------------------------- | ------------------------------------------ | -| Source of Truth | **InfraHub** | Intent definition via custom schema | -| Orchestrator | **Prefect** | Python-native workflow orchestration | -| Transport | gNMI | Configuration and telemetry | -| Data Models | YANG (OpenConfig + Arista) | Structured configuration | -| Python Library | pygnmi + infrahub-sdk | gNMI/InfraHub interactions | -| CLI | Click + Rich | YANG discovery tools | -| Validation | Pydantic v2 | Intent data validation | -| Lab | ContainerLab + cEOS | Development environment | +| Component | Technology | Purpose | +| --------------- | -------------------------- | ------------------------------------ | +| Source of Truth | **InfraHub** | Intent definition via custom schema | +| Orchestrator | **Prefect** | Python-native workflow orchestration | +| Transport | gNMI | Configuration and telemetry | +| Data Models | YANG (OpenConfig + Arista) | Structured configuration | +| Python Library | pygnmi + infrahub-sdk | gNMI/InfraHub interactions | +| CLI | Click + Rich | YANG discovery tools | +| Validation | Pydantic v2 | Intent data validation | +| Lab | ContainerLab + cEOS | Development environment | ## 🔗 Related Projects @@ -136,21 +136,25 @@ fabric-orchestrator/ ## 📚 References ### InfraHub + - [InfraHub Documentation](https://docs.infrahub.app) - [InfraHub Schema Guide](https://docs.infrahub.app/guides/create-schema) - [InfraHub Python SDK](https://github.com/opsmill/infrahub-sdk-python) ### Prefect + - [Prefect Documentation](https://docs.prefect.io) - [Prefect Flows](https://docs.prefect.io/latest/develop/write-flows/) - [Prefect Tasks](https://docs.prefect.io/latest/develop/write-tasks/) ### YANG / gNMI + - [Arista gNMI Documentation](https://aristanetworks.github.io/openmgmt/configuration/gnmi/) - [OpenConfig Models](https://github.com/openconfig/public) - [pygnmi Library](https://github.com/akarneliuk/pygnmi) ### EVPN-VXLAN + - [Arista BGP EVPN Configuration Example](https://overlaid.net/2019/01/27/arista-bgp-evpn-configuration-example/) ## 🚀 Getting Started