Update README.md

Reviewed-on: #2

README.md

@@ -1,6 +1,6 @@
 # Prefect Deployment
 
-GitOps deployment for Prefect workflow orchestration with Tailscale HTTPS access.
+GitOps deployment for Prefect workflow orchestration with Tailscale HTTPS access via Komodo.
 
 ## Prerequisites
 
@@ -13,79 +13,118 @@ CREATE DATABASE prefect;
 CREATE USER prefect WITH PASSWORD 'your-secure-password';
 GRANT ALL PRIVILEGES ON DATABASE prefect TO prefect;
 
--- PostgreSQL 15+ requires:
 \c prefect
+CREATE EXTENSION IF NOT EXISTS pg_trgm;
 GRANT ALL ON SCHEMA public TO prefect;
-```
+ALTER SCHEMA public OWNER TO prefect;
-
-### Host Configuration
-
-Download configuration files to `/opt/prefect` (first time only):
-
-```bash
-sudo mkdir -p /opt/prefect/tailscale
-sudo curl -o /opt/prefect/tailscale/serve-config.json https://gitea.arnodo.fr/Damien/prefect-deployment/raw/branch/main/serve-config.json
 ```
 
 ### Tailscale Auth Key
 
 Generate a reusable auth key from https://login.tailscale.com/admin/settings/keys
 
-## Deployment
+## Deployment with Komodo
 
-1. Create a new stack in Portainer
+### 1. Add Git Provider (if using private repo)
-2. Select "Repository" and point to this repository
+
-3. Portainer will load `stack.env` automatically
+In Komodo UI: Settings → Git Providers → Add your Gitea instance credentials.
-4. Override sensitive values (`CHANGE_ME`) in the environment variables section:
+
-   - `TS_AUTHKEY` - Tailscale auth key (reusable recommended)
+### 2. Create Stack
-   - `DB_PASSWORD` - PostgreSQL password
+
-5. Deploy
+1. Navigate to **Stacks** → **New Stack**
+2. Configure:
+   - **Name**: `prefect`
+   - **Server**: Select your target server
+   - **Source**: Git Repo
+   - **Git Provider**: `gitea.arnodo.fr` (or your provider)
+   - **Repo**: `Damien/prefect-deployment`
+   - **Branch**: `main`
+   - **File Paths**: `docker-compose.yml`
+
+### 3. Configure Environment Variables
+
+In the stack configuration, add the following environment variables:
+
+| Variable | Description | Example |
+|----------|-------------|---------|
+| `TS_AUTHKEY` | Tailscale auth key (reusable) | `tskey-auth-xxx` |
+| `DB_HOST` | PostgreSQL host | `postgresql.taila5ad8.ts.net` |
+| `DB_PORT` | PostgreSQL port | `5432` |
+| `DB_USER` | Database user | `prefect` |
+| `DB_PASSWORD` | Database password | *secret* |
+
+> **Tip**: Use Komodo's secret variables (marked with 🔒) for sensitive values.
+
+### 4. Deploy
+
+Click **Deploy** in Komodo. The stack will clone the repository and start all services.
+
+## GitOps Workflow
+
+### Auto-deploy on Git Push
+
+1. In Komodo, go to your stack settings
+2. Enable **Auto Deploy** on push
+3. Copy the webhook URL
+4. Add it to your Gitea repository: Settings → Webhooks
 
 ## Access
 
 Once deployed: https://prefect.taila5ad8.ts.net
 
-## Architecture
+## Services
 
-```
+| Service | Description | Image |
-┌─────────────────────────────────────────────────────────────┐
+|---------|-------------|-------|
-│                     Tailscale Network                       │
+| `tailscale` | HTTPS ingress via Tailscale | `tailscale/tailscale` |
-├─────────────────────────────────────────────────────────────┤
+| `redis` | Messaging broker | `redis:7-alpine` |
-│                                                             │
+| `prefect-server` | API + UI | `prefecthq/prefect:3-latest` |
-│  ┌─────────────┐     ┌─────────────┐     ┌──────────────┐  │
+| `prefect-services` | Background services | `prefecthq/prefect:3-latest` |
-│  │  Tailscale  │────▶│   Prefect   │────▶│  PostgreSQL  │  │
+| `prefect-worker-pg-backup` | Worker for PostgreSQL backups | `gitea.arnodo.fr/Damien/prefect-flows-pg-backup` |
-│  │   Sidecar   │     │   Server    │     │  (external)  │  │
+
-│  │   :443      │     │   :4200     │     │              │  │
+## Work Pools
-│  └─────────────┘     └──────┬──────┘     └──────────────┘  │
+
-│                             │                               │
+The `prefect-worker-pg-backup` service automatically creates and listens to the `pg-backup-pool` work pool (type: process).
-│                      ┌──────▼──────┐                       │
+
-│                      │    Redis    │                       │
+## Secrets Management
-│                      │  (messaging)│                       │
+
-│                      └─────────────┘                       │
+Flow-specific secrets (S3 credentials, database passwords, API keys, etc.) should be managed via **Prefect Blocks**, not environment variables in the compose file.
-│                                                             │
+
-│                      ┌─────────────┐                       │
+### Creating a Block (example with S3/Garage)
-│                      │   Prefect   │                       │
+
-│                      │  Services   │                       │
+```python
-│                      │ (background)│                       │
+from prefect_aws import AwsCredentials
-│                      └─────────────┘                       │
+
-│                                                             │
+creds = AwsCredentials(
-└─────────────────────────────────────────────────────────────┘
+    aws_access_key_id="xxx",
+    aws_secret_access_key="xxx",
+    aws_endpoint_url="https://s3.taila5ad8.ts.net"
+)
+creds.save("garage-credentials")
 ```
 
-## Directory Structure
+### Using in a flow
 
-```
+```python
-/opt/prefect/
+from prefect import flow
-└── tailscale/
+from prefect_aws import AwsCredentials
-    └── serve-config.json     # Tailscale HTTPS configuration
+
+@flow
+def my_backup_flow():
+    creds = AwsCredentials.load("garage-credentials")
+    # use creds...
+
+my_backup_flow.deploy(
+    name="my-backup",
+    work_pool_name="pg-backup-pool"
+)
 ```
 
-## Post-Deployment: Create Work Pool
+## Repository Structure
 
-After the server is running, create a work pool for your workflows:
-
-```bash
-export PREFECT_API_URL=https://prefect.taila5ad8.ts.net/api
-prefect work-pool create backup-pool --type process
 ```
-
+prefect-deployment/
+├── docker-compose.yml     # Stack definition (relative paths)
+├── serve-config.json      # Tailscale HTTPS serve config
+└── README.md
+```

docker-compose.yml

@@ -10,7 +10,7 @@ services:
       - TS_SERVE_CONFIG=/config/serve-config.json
     volumes:
       - tailscale-state:/var/lib/tailscale
-      - /opt/prefect/tailscale/serve-config.json:/config/serve-config.json:ro
+      - ./serve-config.json:/config/serve-config.json:ro
     cap_add:
       - NET_ADMIN
       - SYS_MODULE
@@ -70,6 +70,18 @@ services:
     command: prefect server services start
     restart: unless-stopped
 
+  # === PREFECT WORKER - PostgreSQL Backup ===
+  prefect-worker-pg-backup:
+    image: gitea.arnodo.fr/damien/prefect-worker-pg-backup:latest
+    container_name: prefect-worker-pg-backup
+    network_mode: service:tailscale
+    depends_on:
+      - prefect-server
+    environment:
+      - PREFECT_API_URL=http://localhost:4200/api
+    command: prefect worker start --pool pg-backup-pool --type process
+    restart: unless-stopped
+
 volumes:
   tailscale-state:
   redis-data:

stack.env

@@ -1,15 +0,0 @@
-# ==============================================================================
-# Prefect Stack Environment Variables
-# ==============================================================================
-# This file is used by Portainer when deploying from Git repository.
-# Sensitive values (marked CHANGE_ME) must be set in Portainer UI.
-# ==============================================================================
-
-# Tailscale
-TS_AUTHKEY=CHANGE_ME
-
-# PostgreSQL (external database)
-DB_HOST=postgresql.taila5ad8.ts.net
-DB_PORT=5432
-DB_USER=prefect
-DB_PASSWORD=CHANGE_ME