prefect-deployment/README.md

Prefect Deployment

GitOps deployment for Prefect workflow orchestration with Tailscale HTTPS access.

Prerequisites

PostgreSQL Database

Connect to your PostgreSQL server and create the database:

CREATE DATABASE prefect;
CREATE USER prefect WITH PASSWORD 'your-secure-password';
GRANT ALL PRIVILEGES ON DATABASE prefect TO prefect;

-- PostgreSQL 15+ requires:
\c prefect
GRANT ALL ON SCHEMA public TO prefect;

Host Configuration

Download configuration files to /opt/prefect (first time only):

sudo mkdir -p /opt/prefect/tailscale
sudo curl -o /opt/prefect/tailscale/serve-config.json https://gitea.arnodo.fr/Damien/prefect-deployment/raw/branch/main/serve-config.json

Tailscale Auth Key

Generate a reusable auth key from https://login.tailscale.com/admin/settings/keys

Deployment

1. Create a new stack in Portainer
2. Select "Repository" and point to this repository
3. Portainer will load stack.env automatically
4. Override sensitive values (CHANGE_ME) in the environment variables section:
   • TS_AUTHKEY - Tailscale auth key (reusable recommended)
   • DB_PASSWORD - PostgreSQL password
5. Deploy

Access

Once deployed: https://prefect.taila5ad8.ts.net

Architecture

┌─────────────────────────────────────────────────────────────┐
│                     Tailscale Network                       │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│  ┌─────────────┐     ┌─────────────┐     ┌──────────────┐  │
│  │  Tailscale  │────▶│   Prefect   │────▶│  PostgreSQL  │  │
│  │   Sidecar   │     │   Server    │     │  (external)  │  │
│  │   :443      │     │   :4200     │     │              │  │
│  └─────────────┘     └──────┬──────┘     └──────────────┘  │
│                             │                               │
│                      ┌──────▼──────┐                       │
│                      │    Redis    │                       │
│                      │  (messaging)│                       │
│                      └─────────────┘                       │
│                                                             │
│                      ┌─────────────┐                       │
│                      │   Prefect   │                       │
│                      │  Services   │                       │
│                      │ (background)│                       │
│                      └─────────────┘                       │
│                                                             │
└─────────────────────────────────────────────────────────────┘

Directory Structure

/opt/prefect/
└── tailscale/
    └── serve-config.json     # Tailscale HTTPS configuration

Post-Deployment: Create Work Pool

After the server is running, create a work pool for your workflows:

export PREFECT_API_URL=https://prefect.taila5ad8.ts.net/api
prefect work-pool create backup-pool --type process