diff --git a/README.md b/README.md index 8519d00..d87b8dd 100644 --- a/README.md +++ b/README.md @@ -51,9 +51,6 @@ In the stack configuration, add the following environment variables: | `DB_PORT` | PostgreSQL port | `5432` | | `DB_USER` | Database user | `prefect` | | `DB_PASSWORD` | Database password | *secret* | -| `S3_ACCESS_KEY` | Garage S3 access key | *secret* | -| `S3_SECRET_KEY` | Garage S3 secret key | *secret* | -| `S3_ENDPOINT_URL` | Garage S3 endpoint | `https://s3.taila5ad8.ts.net` | > **Tip**: Use Komodo's secret variables (marked with 🔒) for sensitive values. @@ -88,14 +85,33 @@ Once deployed: https://prefect.taila5ad8.ts.net The `prefect-worker-pg-backup` service automatically creates and listens to the `pg-backup-pool` work pool (type: process). -To deploy a flow to this pool: +## Secrets Management + +Flow-specific secrets (S3 credentials, database passwords, API keys, etc.) should be managed via **Prefect Blocks**, not environment variables in the compose file. + +### Creating a Block (example with S3/Garage) + +```python +from prefect_aws import AwsCredentials + +creds = AwsCredentials( + aws_access_key_id="xxx", + aws_secret_access_key="xxx", + aws_endpoint_url="https://s3.taila5ad8.ts.net" +) +creds.save("garage-credentials") +``` + +### Using in a flow ```python from prefect import flow +from prefect_aws import AwsCredentials @flow def my_backup_flow(): - ... + creds = AwsCredentials.load("garage-credentials") + # use creds... my_backup_flow.deploy( name="my-backup", diff --git a/docker-compose.yml b/docker-compose.yml index 7043ae9..5c476ce 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -78,12 +78,7 @@ services: depends_on: - prefect-server environment: - # Prefect API connection (via Tailscale) - PREFECT_API_URL=http://localhost:4200/api - # S3 credentials for Garage - - AWS_ACCESS_KEY_ID=${S3_ACCESS_KEY} - - AWS_SECRET_ACCESS_KEY=${S3_SECRET_KEY} - - AWS_ENDPOINT_URL=${S3_ENDPOINT_URL} command: prefect worker start --pool pg-backup-pool --type process restart: unless-stopped