docs: add comprehensive README
This commit is contained in:
94
README.md
Normal file
94
README.md
Normal file
@@ -0,0 +1,94 @@
|
|||||||
|
# Prefect Deployment
|
||||||
|
|
||||||
|
GitOps deployment for Prefect workflow orchestration with Tailscale HTTPS access.
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
### PostgreSQL Database
|
||||||
|
|
||||||
|
Connect to your PostgreSQL server and create the database:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
CREATE DATABASE prefect;
|
||||||
|
CREATE USER prefect WITH PASSWORD 'your-secure-password';
|
||||||
|
GRANT ALL PRIVILEGES ON DATABASE prefect TO prefect;
|
||||||
|
|
||||||
|
-- PostgreSQL 15+ requires:
|
||||||
|
\c prefect
|
||||||
|
GRANT ALL ON SCHEMA public TO prefect;
|
||||||
|
```
|
||||||
|
|
||||||
|
### Host Configuration
|
||||||
|
|
||||||
|
Download configuration files to `/opt/prefect` (first time only):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo mkdir -p /opt/prefect/tailscale
|
||||||
|
sudo curl -o /opt/prefect/tailscale/serve-config.json https://gitea.arnodo.fr/Damien/prefect-deployment/raw/branch/main/serve-config.json
|
||||||
|
```
|
||||||
|
|
||||||
|
### Tailscale Auth Key
|
||||||
|
|
||||||
|
Generate a reusable auth key from https://login.tailscale.com/admin/settings/keys
|
||||||
|
|
||||||
|
## Deployment
|
||||||
|
|
||||||
|
1. Create a new stack in Portainer
|
||||||
|
2. Select "Repository" and point to this repository
|
||||||
|
3. Portainer will load `stack.env` automatically
|
||||||
|
4. Override sensitive values (`CHANGE_ME`) in the environment variables section:
|
||||||
|
- `TS_AUTHKEY` - Tailscale auth key (reusable recommended)
|
||||||
|
- `DB_PASSWORD` - PostgreSQL password
|
||||||
|
5. Deploy
|
||||||
|
|
||||||
|
## Access
|
||||||
|
|
||||||
|
Once deployed: https://prefect.taila5ad8.ts.net
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
```
|
||||||
|
┌─────────────────────────────────────────────────────────────┐
|
||||||
|
│ Tailscale Network │
|
||||||
|
├─────────────────────────────────────────────────────────────┤
|
||||||
|
│ │
|
||||||
|
│ ┌─────────────┐ ┌─────────────┐ ┌──────────────┐ │
|
||||||
|
│ │ Tailscale │────▶│ Prefect │────▶│ PostgreSQL │ │
|
||||||
|
│ │ Sidecar │ │ Server │ │ (external) │ │
|
||||||
|
│ │ :443 │ │ :4200 │ │ │ │
|
||||||
|
│ └─────────────┘ └──────┬──────┘ └──────────────┘ │
|
||||||
|
│ │ │
|
||||||
|
│ ┌──────▼──────┐ │
|
||||||
|
│ │ Redis │ │
|
||||||
|
│ │ (messaging)│ │
|
||||||
|
│ └─────────────┘ │
|
||||||
|
│ │
|
||||||
|
│ ┌─────────────┐ │
|
||||||
|
│ │ Prefect │ │
|
||||||
|
│ │ Services │ │
|
||||||
|
│ │ (background)│ │
|
||||||
|
│ └─────────────┘ │
|
||||||
|
│ │
|
||||||
|
└─────────────────────────────────────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
## Directory Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
/opt/prefect/
|
||||||
|
└── tailscale/
|
||||||
|
└── serve-config.json # Tailscale HTTPS configuration
|
||||||
|
```
|
||||||
|
|
||||||
|
## Post-Deployment: Create Work Pool
|
||||||
|
|
||||||
|
After the server is running, create a work pool for your workflows:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
export PREFECT_API_URL=https://prefect.taila5ad8.ts.net/api
|
||||||
|
prefect work-pool create backup-pool --type process
|
||||||
|
```
|
||||||
|
|
||||||
|
## Next Steps
|
||||||
|
|
||||||
|
See the [flows/](flows/) directory for workflow examples (Phase 2).
|
||||||
Reference in New Issue
Block a user