Merge pull request 'refactor: remove worker, use Prefect Blocks for secrets' (#2) from refactor/remove-worker-use-blocks into main

Reviewed-on: #2
This commit was merged in pull request #2.
This commit is contained in:
2026-01-31 15:11:04 +00:00
2 changed files with 21 additions and 10 deletions

View File

@@ -51,9 +51,6 @@ In the stack configuration, add the following environment variables:
| `DB_PORT` | PostgreSQL port | `5432` |
| `DB_USER` | Database user | `prefect` |
| `DB_PASSWORD` | Database password | *secret* |
| `S3_ACCESS_KEY` | Garage S3 access key | *secret* |
| `S3_SECRET_KEY` | Garage S3 secret key | *secret* |
| `S3_ENDPOINT_URL` | Garage S3 endpoint | `https://s3.taila5ad8.ts.net` |
> **Tip**: Use Komodo's secret variables (marked with 🔒) for sensitive values.
@@ -88,14 +85,33 @@ Once deployed: https://prefect.taila5ad8.ts.net
The `prefect-worker-pg-backup` service automatically creates and listens to the `pg-backup-pool` work pool (type: process).
To deploy a flow to this pool:
## Secrets Management
Flow-specific secrets (S3 credentials, database passwords, API keys, etc.) should be managed via **Prefect Blocks**, not environment variables in the compose file.
### Creating a Block (example with S3/Garage)
```python
from prefect_aws import AwsCredentials
creds = AwsCredentials(
aws_access_key_id="xxx",
aws_secret_access_key="xxx",
aws_endpoint_url="https://s3.taila5ad8.ts.net"
)
creds.save("garage-credentials")
```
### Using in a flow
```python
from prefect import flow
from prefect_aws import AwsCredentials
@flow
def my_backup_flow():
...
creds = AwsCredentials.load("garage-credentials")
# use creds...
my_backup_flow.deploy(
name="my-backup",

View File

@@ -78,12 +78,7 @@ services:
depends_on:
- prefect-server
environment:
# Prefect API connection (via Tailscale)
- PREFECT_API_URL=http://localhost:4200/api
# S3 credentials for Garage
- AWS_ACCESS_KEY_ID=${S3_ACCESS_KEY}
- AWS_SECRET_ACCESS_KEY=${S3_SECRET_KEY}
- AWS_ENDPOINT_URL=${S3_ENDPOINT_URL}
command: prefect worker start --pool pg-backup-pool --type process
restart: unless-stopped