[Phase 4] Create Kestra workflow for fabric reconciliation (plan/apply) #10

New Issue

Damien · 2025-12-20T15:42:19Z

Damien commented

2025-12-20 15:42:19 +00:00

Description

⚠️ Mise à jour : Suite à la migration vers Kestra, les commandes CLI plan et apply sont remplacées par des workflows Kestra. Le CLI conserve uniquement les commandes discover pour l'exploration YANG interactive.

Créer le workflow Kestra principal fabric-reconcile.yml qui orchestre la réconciliation de la fabric.

Workflow: `fabric-reconcile.yml`

id: fabric-reconcile
namespace: network.fabric
description: Reconcile fabric state with NetBox intent (plan/apply)

inputs:
  - id: device
    type: STRING
    required: false
    description: Specific device to reconcile (empty = all devices)
  - id: auto_apply
    type: BOOLEAN
    defaults: false
    description: Automatically apply changes without confirmation
  - id: dry_run
    type: BOOLEAN
    defaults: true
    description: Show plan without applying

tasks:
  - id: get_intent
    type: io.kestra.plugin.scripts.python.Script
    containerImage: ghcr.io/damien/fabric-orchestrator:latest
    script: |
      from kestra import Kestra
      from src.netbox import FabricNetBoxClient
      
      client = FabricNetBoxClient()
      if "{{ inputs.device }}":
          intent = client.get_device_intent("{{ inputs.device }}")
      else:
          intent = client.get_fabric_intent()
      
      Kestra.outputs({"intent": intent})

  - id: get_current_state
    type: io.kestra.plugin.scripts.python.Script
    containerImage: ghcr.io/damien/fabric-orchestrator:latest
    script: |
      from kestra import Kestra
      from src.gnmi import GNMIClient
      # Get current state from devices via gNMI Get
      Kestra.outputs({"current_state": state})

  - id: compute_diff
    type: io.kestra.plugin.scripts.python.Script
    containerImage: ghcr.io/damien/fabric-orchestrator:latest
    script: |
      from kestra import Kestra
      from src.reconciler.diff import compute_diff
      
      changes = compute_diff(
          want="{{ outputs.get_intent.vars.intent }}",
          have="{{ outputs.get_current_state.vars.current_state }}"
      )
      
      Kestra.outputs({
          "changes": changes,
          "has_changes": len(changes) > 0,
          "summary": f"{len(changes)} changes detected"
      })

  - id: log_plan
    type: io.kestra.plugin.core.log.Log
    message: |
      📋 Execution Plan
      {{ outputs.compute_diff.vars.summary }}
      
      Changes:
      {{ outputs.compute_diff.vars.changes | json }}

  - id: apply_changes
    type: io.kestra.plugin.scripts.python.Script
    runIf: "{{ outputs.compute_diff.vars.has_changes and inputs.auto_apply and not inputs.dry_run }}"
    containerImage: ghcr.io/damien/fabric-orchestrator:latest
    script: |
      from kestra import Kestra
      from src.gnmi import GNMIClient
      # Apply changes via gNMI Set
      Kestra.outputs({"applied": True, "results": results})

triggers:
  - id: manual
    type: io.kestra.plugin.core.trigger.Schedule
    disabled: true  # Manual trigger only

errors:
  - id: notify_failure
    type: io.kestra.plugin.notifications.slack.SlackExecution
    url: "{{ secret('SLACK_WEBHOOK') }}"

Tasks

Créer le fichier kestra/flows/fabric-reconcile.yml
Implémenter la tâche get_intent (appel NetBox client)
Implémenter la tâche get_current_state (appel gNMI Get)
Implémenter la tâche compute_diff (diff engine)
Implémenter la tâche apply_changes (gNMI Set)
Ajouter gestion des erreurs et notifications
Tester le workflow en mode dry-run
Tester le workflow avec auto_apply

Équivalence avec l'ancien CLI

Ancienne commande	Nouveau workflow Kestra
`fabric-orch plan`	Exécuter `fabric-reconcile` avec `dry_run: true`
`fabric-orch apply`	Exécuter `fabric-reconcile` avec `auto_apply: true`
`fabric-orch plan --device leaf1`	Input `device: leaf1`

Output

kestra/flows/fabric-reconcile.yml

Dependencies

#7 (Diff engine)
#8 (gNMI Get)
#9 (gNMI Set)

## Description > ⚠️ **Mise à jour** : Suite à la migration vers Kestra, les commandes CLI `plan` et `apply` sont remplacées par des **workflows Kestra**. Le CLI conserve uniquement les commandes `discover` pour l'exploration YANG interactive. Créer le workflow Kestra principal `fabric-reconcile.yml` qui orchestre la réconciliation de la fabric. ## Workflow: `fabric-reconcile.yml` ```yaml id: fabric-reconcile namespace: network.fabric description: Reconcile fabric state with NetBox intent (plan/apply) inputs: - id: device type: STRING required: false description: Specific device to reconcile (empty = all devices) - id: auto_apply type: BOOLEAN defaults: false description: Automatically apply changes without confirmation - id: dry_run type: BOOLEAN defaults: true description: Show plan without applying tasks: - id: get_intent type: io.kestra.plugin.scripts.python.Script containerImage: ghcr.io/damien/fabric-orchestrator:latest script: | from kestra import Kestra from src.netbox import FabricNetBoxClient client = FabricNetBoxClient() if "{{ inputs.device }}": intent = client.get_device_intent("{{ inputs.device }}") else: intent = client.get_fabric_intent() Kestra.outputs({"intent": intent}) - id: get_current_state type: io.kestra.plugin.scripts.python.Script containerImage: ghcr.io/damien/fabric-orchestrator:latest script: | from kestra import Kestra from src.gnmi import GNMIClient # Get current state from devices via gNMI Get Kestra.outputs({"current_state": state}) - id: compute_diff type: io.kestra.plugin.scripts.python.Script containerImage: ghcr.io/damien/fabric-orchestrator:latest script: | from kestra import Kestra from src.reconciler.diff import compute_diff changes = compute_diff( want="{{ outputs.get_intent.vars.intent }}", have="{{ outputs.get_current_state.vars.current_state }}" ) Kestra.outputs({ "changes": changes, "has_changes": len(changes) > 0, "summary": f"{len(changes)} changes detected" }) - id: log_plan type: io.kestra.plugin.core.log.Log message: | 📋 Execution Plan {{ outputs.compute_diff.vars.summary }} Changes: {{ outputs.compute_diff.vars.changes | json }} - id: apply_changes type: io.kestra.plugin.scripts.python.Script runIf: "{{ outputs.compute_diff.vars.has_changes and inputs.auto_apply and not inputs.dry_run }}" containerImage: ghcr.io/damien/fabric-orchestrator:latest script: | from kestra import Kestra from src.gnmi import GNMIClient # Apply changes via gNMI Set Kestra.outputs({"applied": True, "results": results}) triggers: - id: manual type: io.kestra.plugin.core.trigger.Schedule disabled: true # Manual trigger only errors: - id: notify_failure type: io.kestra.plugin.notifications.slack.SlackExecution url: "{{ secret('SLACK_WEBHOOK') }}" ``` ## Tasks - [ ] Créer le fichier `kestra/flows/fabric-reconcile.yml` - [ ] Implémenter la tâche `get_intent` (appel NetBox client) - [ ] Implémenter la tâche `get_current_state` (appel gNMI Get) - [ ] Implémenter la tâche `compute_diff` (diff engine) - [ ] Implémenter la tâche `apply_changes` (gNMI Set) - [ ] Ajouter gestion des erreurs et notifications - [ ] Tester le workflow en mode dry-run - [ ] Tester le workflow avec auto_apply ## Équivalence avec l'ancien CLI | Ancienne commande | Nouveau workflow Kestra | |-------------------|-------------------------| | `fabric-orch plan` | Exécuter `fabric-reconcile` avec `dry_run: true` | | `fabric-orch apply` | Exécuter `fabric-reconcile` avec `auto_apply: true` | | `fabric-orch plan --device leaf1` | Input `device: leaf1` | ## Output - `kestra/flows/fabric-reconcile.yml` ## Dependencies - #7 (Diff engine) - #8 (gNMI Get) - #9 (gNMI Set)

Damien added the phase-2-minimal-reconciler label 2025-12-20 15:42:38 +00:00

Damien changed title from ~~[Phase 2] Create CLI with plan and apply commands~~ to [Phase 4] Create Kestra workflow for fabric reconciliation (plan/apply)

2026-01-10 13:00:18 +00:00

Damien referenced this issue

2026-01-10 13:16:22 +00:00

[Phase 4] Implement auto-remediation workflow with Kestra #19

Damien referenced this issue

2026-01-10 13:17:37 +00:00

[Phase 4] Create NetBox webhook trigger workflow #26

Damien referenced this issue

2026-01-10 14:34:24 +00:00

feat: migrate orchestration layer to Kestra #27

Sign in to join this conversation.