proxy/README.md

@@ -60,3 +60,42 @@ sudo ufw delete allow 22/tcp
 - Access NPM admin: `https://proxy.<your-tailnet>.ts.net`
 - Default credentials: `admin@example.com` / `changeme`
 - Optionally approve exit-node in Tailscale admin console
+
+## Centralized log reception (rsyslog)
+
+A fail2ban jail running inside an exposed service's own LXC only ever sees
+this proxy's tailnet IP as the connection source, so it would end up
+banning the proxy itself instead of the actual client. Detection has to
+stay where the signal is (the service's application log); banning has to
+happen here, where public connections terminate. Services forward their
+logs to this proxy over TCP so a jail here can act on them.
+
+| File | Purpose |
+|------|---------|
+| `/etc/rsyslog.d/10-remote-receiver.conf` | Generic `imtcp` listener, port `RSYSLOG_PORT` (default `5514`). Anything not claimed by a more specific routing file lands in `/var/log/remote/<sender-hostname>.log`. |
+| `/etc/rsyslog.d/50-<service>.conf` | One per exposed service. Routes by tag/programname into that service's own logfile for its dedicated fail2ban jail. |
+| `/etc/logrotate.d/remote-logs` | Rotation for everything under `/var/log/remote/` (`copytruncate`, so fail2ban never loses its file descriptor across a rotation). |
+
+Adding a new exposed service is a matter of dropping its `50-<service>.conf`
+here — nothing else in this list needs to change. A minimal example that
+routes messages tagged `myservice` into their own file, in addition to the
+generic catch-all:
+
+```
+$RuleSet remoteLogs
+if $programname == 'myservice' then {
+    action(type="omfile" file="/var/log/myservice/myservice.log")
+}
+$RuleSet RSYSLOG_DefaultRuleset
+```
+
+`RSYSLOG_PORT` and `RSYSLOG_BIND_ADDR` (default `0.0.0.0`) are overridable
+via environment. The default bind is safe as-is: UFW's default-deny only
+opens `80/tcp` and `443/tcp` publicly, so port `5514` is reachable
+exclusively over `tailscale0` regardless of the bind address. Binding to
+the tailnet IP directly was considered and rejected — it would require
+`tailscale up` to have already succeeded before rsyslog is configured,
+which complicates the script's flow (a missing `TS_AUTHKEY` is tolerated
+today). The residual risk is log injection from anything that reaches the
+port (which can trigger a false fail2ban ban); narrow `RSYSLOG_BIND_ADDR`
+to a specific tailnet IP if that risk becomes a concern.

proxy/install.sh

@@ -44,6 +44,13 @@ ACME_EMAIL="${ACME_EMAIL:-}"
 # Generate at https://login.tailscale.com/admin/settings/keys
 TS_AUTHKEY="${TS_AUTHKEY:-}"
 
+# rsyslog receiver for exposed services' logs (see "Configuring rsyslog" below).
+RSYSLOG_PORT="${RSYSLOG_PORT:-5514}"
+# Default 0.0.0.0 is fine: UFW's default-deny only opens 80/443 publicly, so
+# this port is reachable exclusively over tailscale0 either way. Override to
+# a specific tailnet IP to shrink the blast radius of log injection instead.
+RSYSLOG_BIND_ADDR="${RSYSLOG_BIND_ADDR:-0.0.0.0}"
+
 main() {
     log_info "=== Proxy Server Deployment (Traefik v3) ==="
 
@@ -75,7 +82,7 @@ main() {
 
     log_info "Installing base packages..."
     sudo apt update -qq
-    sudo apt install -y -qq vim ca-certificates curl gnupg lsb-release fail2ban unattended-upgrades ufw ethtool networkd-dispatcher > /dev/null
+    sudo apt install -y -qq vim ca-certificates curl gnupg lsb-release fail2ban unattended-upgrades ufw ethtool networkd-dispatcher rsyslog > /dev/null
 
     log_info "Installing Tailscale..."
     curl -fsSL https://tailscale.com/install.sh | sh
@@ -174,6 +181,40 @@ EOF
 
     sudo systemctl restart fail2ban
 
+    log_info "Configuring rsyslog receiver for exposed services..."
+    # A fail2ban jail running inside a service's own LXC only ever sees this
+    # proxy's tailnet IP as the source of connections, so it would end up
+    # banning the proxy itself. Detection has to stay where the signal is
+    # (the service's application log); banning has to happen here, at the
+    # edge where public connections actually terminate. Services forward
+    # their logs to this receiver over TCP; adding a new one is a matter of
+    # dropping a 50-<service>.conf here (see proxy/README.md) — nothing else
+    # to touch.
+    sudo mkdir -p /var/log/remote
+    sudo tee /etc/rsyslog.d/10-remote-receiver.conf > /dev/null << EOF
+module(load="imtcp")
+
+\$RuleSet remoteLogs
+\$template RemoteLogPath,"/var/log/remote/%HOSTNAME%.log"
+*.* ?RemoteLogPath
+\$RuleSet RSYSLOG_DefaultRuleset
+
+input(type="imtcp" port="${RSYSLOG_PORT}" address="${RSYSLOG_BIND_ADDR}" ruleset="remoteLogs")
+EOF
+
+    sudo tee /etc/logrotate.d/remote-logs > /dev/null << 'EOF'
+/var/log/remote/*.log {
+    daily
+    rotate 7
+    compress
+    missingok
+    notifempty
+    copytruncate
+}
+EOF
+
+    sudo systemctl restart rsyslog
+
     log_info "Creating Traefik stack under $TRAEFIK_DIR..."
     mkdir -p "$TRAEFIK_DIR/conf.d"