n8n Credential Portability — Playbook & Standards

Covers: bms-4 n8n (queue mode, PostgreSQL, 2.26.3) Secrets management: SOPS + age — see docs/secrets-management.md Companion to: n8n-operations.md


1. Honest Weak Point Analysis

Weak Point 1 (CRITICAL): N8N_ENCRYPTION_KEY is NOT passed to containers

What’s happening: N8N_ENCRYPTION_KEY is absent from every environment block in bms-4/docker-compose.yml. n8n auto-generates a random key on first start and stores it in /home/node/.n8n/config inside the n8n_data Docker volume.

Consequences:

  • If n8n_data volume is lost → encryption key is gone → all n8n credentials become unrecoverable permanently
  • The restore doc already acknowledges this: “Credentials must be re-entered manually”

Fix (Phase 1): Extract the current key, add it to secrets/n8n-bms4.env.sops, and add it explicitly to all container environment blocks. Key then travels with every secrets-sync.yml deploy — volume loss no longer kills the credential store.


Weak Point 2 (MEDIUM): $env exposes infrastructure secrets to every workflow

N8N_EXPOSE_ENVIRONMENT_VARIABLES=true + N8N_BLOCK_ENV_ACCESS_IN_NODE=false is set on all containers. This means any workflow can read $env.DB_POSTGRESDB_PASSWORD, $env.REDIS_PASSWORD, $env.GITHUB_PAT_ALL_WRITES, etc.

n8n Community Edition has no selective env var filtering — it’s all-or-nothing.

Accepted risk + mitigations:

  • Only n8n admin can create/import workflows (guard UI + API key)
  • Never import untrusted workflow JSON
  • DB + Redis are internal Docker network only — exploitation requires n8n access first

Weak Point 3 (MEDIUM): Secret values appear in execution logs for 7 days

When $env.ATRAX_PASSWORD flows through an HTTP Request node, n8n stores the evaluated value in execution history. Unlike n8n credential store (which masks values), $env values are not masked.

Mitigation already in place: EXECUTIONS_DATA_PRUNE=true + MAX_AGE=168h (7 days) — auto-purged.

Accepted trade-off: Zero credential re-entry on migration > 7-day log exposure window for this infra use case.


Weak Point 4 (MEDIUM): No Docker volume backup

Neither n8n_data (encryption key) nor n8n_postgres_data (execution history + credential vault) is backed up. Nightly GH Action backs up workflow JSON + credential metadata only.

Fix (Phase 3): Weekly pg_dump n8n to Wasabi. Once N8N_ENCRYPTION_KEY is in SOPS, n8n_data becomes non-critical.


Weak Point 5 (LOW): OAuth credentials don’t survive domain change

If n8n moves to a new domain, OAuth callback URLs change and token refresh fails at the provider. Currently no OAuth credentials in use (LinkedIn uses a static bearer token). Document as a future concern.


Weak Point 6 (LOW): Secret rotation requires container restart (~30s downtime)

LINKEDIN_ACCESS_TOKEN expires every 60 days. Rotation: edit SOPS → commit → push → secrets-sync.ymlstart.sh restarts containers. Watchdogs and retry logic cover the window.


2. Architecture Decision: What Goes Where

Credential typeMechanismPortable?Masked in logs?
API keys, tokens, passwordsn8n credential vault via sync-n8n-credentials.pyYes — if N8N_ENCRYPTION_KEY in SOPSYes — masked in logs
OAuth (future)n8n credential store + shared N8N_ENCRYPTION_KEYYes — if key in SOPSYes
Non-secret config (URLs, IDs)n8n Variables ($vars.VAR)Partial — values must exist on targetN/A
Hardcoded in Set node valuesNeverNoNo

Rule (updated #1231): API keys and tokens in workflow nodes must use the n8n credential vault (httpHeaderAuth) populated by sync-n8n-credentials.py --cred KEY. Using $env.SOMETHING directly in node parameters is permitted only for non-secret config (IDs, URLs) and for secrets not yet migrated. If a node has a literal API key typed in, that is a bug.


3. Implementation Plan

Phase 1 — Fix the encryption key (1h, one-time) ← URGENT

Goal: N8N_ENCRYPTION_KEY lives in secrets/n8n-bms4.env.sops and is explicitly passed to all containers. Volume loss no longer destroys the credential store.

Step 1.1 — Extract current key from running container

ssh root@54.36.123.110
docker exec bms-4-n8n-1 cat /home/node/.n8n/config | python3 -m json.tool | grep encryptionKey

Do NOT paste the key value in chat. Copy it directly into the SOPS editor in Step 1.2.

Step 1.2 — Add to SOPS

# Local dev machine (requires age key at ~/.age/p24-infra-keys.txt)
export SOPS_AGE_KEY_FILE="$HOME/.age/p24-infra-keys.txt"
sops secrets/n8n-bms4.env.sops
# Add line: N8N_ENCRYPTION_KEY=<extracted value>
# Save and exit — SOPS re-encrypts automatically

Step 1.3 — Add to docker-compose.yml

In bms-4/docker-compose.yml, add to the environment: block of n8n, n8n-worker-1, n8n-worker-2, n8n-worker-3 (after EXECUTIONS_MODE=queue):

- N8N_ENCRYPTION_KEY=${N8N_ENCRYPTION_KEY}

Step 1.4 — Commit and push to main

git add secrets/n8n-bms4.env.sops bms-4/docker-compose.yml
git commit -m "fix(#789): pin N8N_ENCRYPTION_KEY from SOPS + add to all n8n containers"
git push origin main
# secrets-sync.yml auto-triggers → deploys .env to bms-4 → start.sh restarts containers

Step 1.5 — Verify

ssh root@54.36.123.110
docker exec bms-4-n8n-1 printenv N8N_ENCRYPTION_KEY
# Must print the key (non-empty)
 
curl -s https://n8n.bms-4.infra.zintegrowana.online/healthz
# Expected: {"status":"ok"}

Phase 2 — Add n8n Variables for non-secret config

BLOCKED — Community Edition limitation. feat:variables requires a paid n8n license. Verified 2026-06-21: GET /api/v1/variables returns HTTP 403 "Your license does not allow for feat:variables".

Workaround: Use n8n “Set” nodes with hardcoded non-secret values (URLs, bucket names), or keep them in $env via SOPS and docker-compose. The $env pattern already covers secrets; for public non-secret config (Supabase URL, Wasabi endpoint), hardcoded values in Set nodes are acceptable — they are not secrets and don’t need rotation.

Skip this phase until an Enterprise license is obtained.


Phase 3 — Add PostgreSQL nightly backup (2h)

Goal: Nightly pg_dump to Wasabi. Combined with N8N_ENCRYPTION_KEY from SOPS, enables full restore with zero manual credential re-entry.

In .github/workflows/n8n-backup.yml, add after existing workflow export step:

- name: Dump n8n PostgreSQL to Wasabi
  env:
    AWS_ACCESS_KEY_ID: ${{ secrets.P24_INFRA_WASABI_ACCESS_KEY }}
    AWS_SECRET_ACCESS_KEY: ${{ secrets.P24_INFRA_WASABI_SECRET_KEY }}
    SSH_KEY: ${{ secrets.VPS_ROOT_SSH_KEY }}
  run: |
    mkdir -p ~/.ssh
    echo "$SSH_KEY" | base64 -d > ~/.ssh/id_bms4 && chmod 600 ~/.ssh/id_bms4
    ssh-keyscan -H 54.36.123.110 >> ~/.ssh/known_hosts
    DATE=$(date +%Y-%m-%d)
    ssh -i ~/.ssh/id_bms4 root@54.36.123.110 \
      "docker exec bms-4-n8n-postgres-1 pg_dump -U n8n n8n | gzip > /tmp/n8n-postgres-${DATE}.sql.gz"
    scp -i ~/.ssh/id_bms4 root@54.36.123.110:/tmp/n8n-postgres-${DATE}.sql.gz /tmp/
    aws s3 cp /tmp/n8n-postgres-${DATE}.sql.gz \
      s3://p24-infra/n8n/postgres-${DATE}.sql.gz \
      --endpoint-url https://s3.eu-central-2.wasabisys.com
    ssh -i ~/.ssh/id_bms4 root@54.36.123.110 "rm /tmp/n8n-postgres-${DATE}.sql.gz"
    rm -f /tmp/n8n-postgres-${DATE}.sql.gz ~/.ssh/id_bms4

Phase 4 — Audit credential store

Audited 2026-06-21. 45 credentials found. Categories:

CategoryCountAction
OAuth 2.0 (Gmail, Google Drive, YouTube, Calendar, WhatsApp)~10Keep in store — OAuth tokens cannot meaningfully be stored as env vars
Telegram bots (service, prod, dev, manager, et-furpac, WAHA)6Keep in store
Supabase / PostgreSQL6Acceptable in store; also covered by $env for workflow expressions
AWS / Wasabi2Covered by $env.WASABI_ACCESS_KEY; credential entries are redundant but harmless
API keys (Anthropic, Groq, OpenAI, Gemini, Cohere, OpenRouter, xAi)~8Keep in store (masked in logs unlike $env)
Webhooks / HMAC / bearer tokens~7Keep in store
SMTP / IMAP4Keep in store (OAuth or password-based)
Other (GitLab, p24ic, basic auth)~2Review when encountered

Finding: The credential store is used correctly. OAuth credentials cannot be replaced with $env. For API key credentials, the store is actually preferable to $env because n8n masks stored credential values in execution logs — $env values are visible in logs for 7 days (see Weak Point 3).

Risk mitigation complete: N8N_ENCRYPTION_KEY is in SOPS + pg_dump runs nightly to Wasabi. Loss of the n8n container volume no longer destroys credentials.


4. Migration Playbook — Moving n8n to a New Server

Pre-flight checklist

  • N8N_ENCRYPTION_KEY is in secrets/n8n-bms4.env.sops (Phase 1 complete)
  • secrets-sync.yml has run successfully on the source server — .env is current
  • Confirm key matches: docker exec bms-4-n8n-1 printenv N8N_ENCRYPTION_KEY
  • Note any OAuth credentials needing re-auth if domain changes

Step 1 — Export workflows

curl -s -H "X-N8N-API-KEY: ${BMS4_N8N_API_KEY}" \
  https://n8n.bms-4.infra.zintegrowana.online/api/v1/workflows \
  > /tmp/n8n-workflows-$(date +%Y-%m-%d).json

Step 2 — Dump PostgreSQL

# On source server
docker exec bms-4-n8n-postgres-1 pg_dump -U n8n n8n \
  | gzip > /tmp/n8n-postgres-export.sql.gz
scp root@54.36.123.110:/tmp/n8n-postgres-export.sql.gz /tmp/

Step 3 — Provision new server

  1. Clone repo: git clone git@github.com:radieu/p24-infra.git /opt/p24-infra
  2. Run secrets-sync.yml targeting new server (or manually SCP the .env from SOPS)
  3. bash /opt/p24-infra/bms-4/start.sh — sources .env (contains N8N_ENCRYPTION_KEY from SOPS) → starts containers
  4. Wait: curl -s http://localhost:5678/healthz

Step 4a — Restore via PostgreSQL dump (preferred)

# On new server
docker exec -i bms-4-n8n-postgres-1 psql -U n8n n8n \
  < <(zcat /tmp/n8n-postgres-export.sql.gz)
docker compose restart n8n n8n-worker-1 n8n-worker-2 n8n-worker-3

Same N8N_ENCRYPTION_KEY from SOPS → all credential values decrypt. Zero manual re-entry.

Step 4b — Restore from Wasabi backup (fallback — no source server)

DATE="YYYY-MM-DD"   # latest available backup
aws s3 cp s3://p24-infra/n8n/postgres-${DATE}.sql.gz /tmp/ \
  --endpoint-url https://s3.eu-central-2.wasabisys.com
docker exec -i bms-4-n8n-postgres-1 psql -U n8n n8n \
  < <(zcat /tmp/n8n-postgres-${DATE}.sql.gz)
docker compose restart n8n n8n-worker-1 n8n-worker-2 n8n-worker-3

Step 5 — Activate workflows and verify

# Activate all workflows
for ID in $(curl -s -H "X-N8N-API-KEY: ${BMS4_N8N_API_KEY}" \
  http://localhost:5678/api/v1/workflows | python3 -c "import sys,json; [print(w['id']) for w in json.load(sys.stdin)['data']]"); do
  curl -s -X PATCH -H "X-N8N-API-KEY: ${BMS4_N8N_API_KEY}" \
    -H "Content-Type: application/json" \
    -d '{"active":true}' "http://localhost:5678/api/v1/workflows/$ID"
done
 
# Check ATRAX GPS sync
curl -s -H "X-N8N-API-KEY: ${BMS4_N8N_API_KEY}" \
  "http://localhost:5678/api/v1/executions?workflowId=CCx9UMdphmGficDX&limit=3" \
  | python3 -m json.tool | grep '"status"'

Step 6 — OAuth re-auth (if domain changed)

If new server runs under a different domain, for each OAuth credential:

  1. Update redirect URI in provider’s OAuth app console to new https://<domain>/rest/oauth2-credential/callback
  2. In n8n UI: Credentials → open → Reconnect → complete OAuth flow

5. New Workflow Authoring Standards

Correct — vault credential (preferred, log-safe)

API keys and tokens must use the n8n credential vault so values are masked in execution logs.

{
  "name": "Call Supabase",
  "type": "n8n-nodes-base.httpRequest",
  "typeVersion": 4.2,
  "parameters": {
    "authentication": "genericCredentialType",
    "genericAuthType": "httpHeaderAuth",
    "url": "https://mwkqmgadqnkkihjdeqsi.supabase.co/rest/v1/my_table",
    "sendHeaders": true,
    "headerParameters": {
      "parameters": [
        { "name": "Authorization", "value": "=\{{ \"Bearer \" + $credentials.httpHeaderAuth.value }}" },
        { "name": "Content-Type",  "value": "application/json" }
      ]
    }
  },
  "credentials": {
    "httpHeaderAuth": { "id": "<cred-id>", "name": "supabase-service-role-key" }
  }
}

Notes:

  • The apikey header is auto-injected by the httpHeaderAuth credential (configured with name: "apikey").
  • Authorization: Bearer references the vault value via $credentials.httpHeaderAuth.value.
  • To push/update the vault value: python3 bms-4/sync-n8n-credentials.py --cred SUPABASE_SERVICE_ROLE_KEY

Acceptable — $env for non-secret config or unmigrated credentials

{
  "name": "Call Supabase (legacy pattern)",
  "type": "n8n-nodes-base.httpRequest",
  "parameters": {
    "headerParameters": {
      "parameters": [
        { "name": "apikey",        "value": "={{ $env.SUPABASE_SERVICE_KEY }}" },
        { "name": "Authorization", "value": "=Bearer {{ $env.SUPABASE_SERVICE_KEY }}" }
      ]
    }
  }
}

Wrong — hardcoded secret

{ "name": "apiKey", "value": "sk-real-key-here", "type": "string" }

Adding a new secret to n8n (vault path)

  1. Add the key to secrets/n8n-bms4.env.sops and register it in CREDENTIALS list in bms-4/sync-n8n-credentials.py
  2. secrets-sync.yml deploys SOPS → bms-4 .env, then run python3 bms-4/sync-n8n-credentials.py --cred MY_KEY on bms-4
  3. In workflow node: set authentication: genericCredentialType, genericAuthType: httpHeaderAuth, add credentials.httpHeaderAuth reference

6. Secret Rotation Schedule

SecretExpiryRotation pathDowntime
LINKEDIN_ACCESS_TOKEN60 dayssops edit secrets/n8n-bms4.env.sops → commit → push~30s
ATRAX_PASSWORDOn demandSame~30s
WASABI_ACCESS_KEY/SECRETQuarterlyWasabi IAM rotation → sops edit → commit → push~30s
N8N_ENCRYPTION_KEYNever (unless compromised)Export credentials first → sops edit → push → verify~5 min
SUPABASE_SERVICE_ROLE_KEYOn demandSOPS edit → commit → push → python3 bms-4/sync-n8n-credentials.py --cred SUPABASE_SERVICE_ROLE_KEY on bms-4Zero — vault PATCH is live

7. What’s Already Good (Don’t Change)

  • n8n credential vault (httpHeaderAuth) for API keys — log-safe, synced via sync-n8n-credentials.py (#1231)
  • EXECUTIONS_DATA_PRUNE=true + 7-day TTL — mitigates log exposure
  • SOPS + age as single source of truth for all n8n secrets (secrets/n8n-bms4.env.sops)
  • Nightly workflow JSON backup to Wasabi
  • All workers share n8n_data volume (encryption key access for workers)
  • N8N_RUNNERS_ENABLED=false (hotfix stable) — don’t touch