86 lines
2.2 KiB
Markdown
86 lines
2.2 KiB
Markdown
# Prefect Deployment
|
|
|
|
GitOps deployment for Prefect workflow orchestration with Tailscale HTTPS access.
|
|
|
|
## Prerequisites
|
|
|
|
### PostgreSQL Database
|
|
|
|
Connect to your PostgreSQL server and create the database:
|
|
|
|
```sql
|
|
CREATE DATABASE prefect;
|
|
CREATE USER prefect WITH PASSWORD 'your-secure-password';
|
|
GRANT ALL PRIVILEGES ON DATABASE prefect TO prefect;
|
|
|
|
-- PostgreSQL 15+ requires:
|
|
\c prefect
|
|
GRANT ALL ON SCHEMA public TO prefect;
|
|
```
|
|
|
|
### Host Configuration
|
|
|
|
Download configuration files to `/opt/prefect` (first time only):
|
|
|
|
```bash
|
|
sudo mkdir -p /opt/prefect/tailscale
|
|
sudo curl -o /opt/prefect/tailscale/serve-config.json https://gitea.arnodo.fr/Damien/prefect-deployment/raw/branch/main/serve-config.json
|
|
```
|
|
|
|
### Tailscale Auth Key
|
|
|
|
Generate a reusable auth key from https://login.tailscale.com/admin/settings/keys
|
|
|
|
## Deployment
|
|
|
|
1. Create a new stack in Portainer
|
|
2. Select "Repository" and point to this repository
|
|
3. Portainer will load `stack.env` automatically
|
|
4. Override sensitive values (`CHANGE_ME`) in the environment variables section:
|
|
- `TS_AUTHKEY` - Tailscale auth key (reusable recommended)
|
|
- `DB_PASSWORD` - PostgreSQL password
|
|
- `S3_ACCESS_KEY` - Garage S3 access key
|
|
- `S3_SECRET_KEY` - Garage S3 secret key
|
|
5. Deploy
|
|
|
|
## Access
|
|
|
|
Once deployed: https://prefect.taila5ad8.ts.net
|
|
|
|
## Services
|
|
|
|
| Service | Description | Image |
|
|
|---------|-------------|-------|
|
|
| `tailscale` | HTTPS ingress via Tailscale | `tailscale/tailscale` |
|
|
| `redis` | Messaging broker | `redis:7-alpine` |
|
|
| `prefect-server` | API + UI | `prefecthq/prefect:3-latest` |
|
|
| `prefect-services` | Background services | `prefecthq/prefect:3-latest` |
|
|
| `prefect-worker-pg-backup` | Worker for PostgreSQL backups | `gitea.arnodo.fr/damien/prefect-worker-pg-backup` |
|
|
|
|
## Work Pools
|
|
|
|
The `prefect-worker-pg-backup` service automatically creates and listens to the `pg-backup-pool` work pool (type: process).
|
|
|
|
To deploy a flow to this pool:
|
|
|
|
```python
|
|
from prefect import flow
|
|
|
|
@flow
|
|
def my_backup_flow():
|
|
...
|
|
|
|
my_backup_flow.deploy(
|
|
name="my-backup",
|
|
work_pool_name="pg-backup-pool"
|
|
)
|
|
```
|
|
|
|
## Directory Structure
|
|
|
|
```
|
|
/opt/prefect/
|
|
└── tailscale/
|
|
└── serve-config.json # Tailscale HTTPS configuration
|
|
```
|