Playbook: BMS4_N8N_API_KEY Rotation

Part of the n8n playbook index (#2641) — Credential rotation category. Consumed by scripts/check-p24-workflows-connection.sh (auth + retry audit, docs/n8n-workflow-inventory.md §Stale API key tracker).

Service: n8n (self-hosted, bms-4) Secret: BMS4_N8N_API_KEY Rotation frequency: 90 days (or immediately on suspected exposure) Last rotated: 2026-08-01 (exposure-triggered, #4918 — Playwright automation, scripts/rotate/n8n-bms4-api-key.js) Next due: 2026-10-30

What uses this credential

Consumer	How	Effect if missing
Monitoring stack (vps-i1)	HTTP calls to n8n REST API for workflow triggers	Automated workflow triggers fail
`credential-exporter`	Checks key age for rotation alerts	Rotation alert suppressed
Scripts in `scripts/`	Direct n8n API calls	Script failures
n8n MCP tools	`HSTGR_N8N_MCP_TOKEN` is separate; this key is for REST API auth	REST API calls 401

Where stored

Location	Key name	How to update
`secrets/n8n-bms4.env.sops`	`BMS4_N8N_API_KEY`	SOPS write pattern (see below)
GH Secret `radieu/p24-infra`	`HSTGR_N8N_API_KEY`	`gh secret set`
bms-4 deployed env	`/opt/p24-infra/bms-4/.env`	Auto-synced by `secrets-sync.yml` on merge to dev/main
bms-4 monitoring env	`/root/.p24-auth-monitor.env`	Manual SSH update — used by p24-auth monitoring cron

Automation status

Playwright-automatable (Tier 2) — Claude can rotate this without human involvement, but requires Playwright MCP (browser session). The n8n Settings → API page allows creating and deleting API keys.

n8n 2.26.3 finding (2026-07-02): The REST API endpoints for API key management (/api/v1/user/api-key, /api/v1/users/{id}/api-keys, etc.) all return 404. The keys are stored as signed JWTs in the user_api_keys DB table — they cannot be replaced via direct DB UPDATE (signature would break). Rotation requires Playwright or manual UI login.

n8n admin password (2026-07-03 confirmed): bms4_n8n_db_password in SOPS is the PostgreSQL password — NOT the n8n web UI admin password. The n8n admin password was originally stored as BMS4_N8N_RADIEU_PASSWORD (renamed 2026-07-09 in #3287 from the shell-invalid key n8n-bms-radieu-password), but that key was superseded 2026-08-01 (#4918) after a self-service password reset created a new key, BMS4_N8N_ADMIN_PASSWORD, in secrets/n8n-bms4.env.sops — that is the current live key (confirmed via live login test, HTTP 200; BMS4_N8N_RADIEU_PASSWORD returns 401 as of 2026-08-11, #6088). scripts/rotate/n8n-bms4-api-key.ps1 already reads BMS4_N8N_ADMIN_PASSWORD, not BMS4_N8N_RADIEU_PASSWORD — this doc previously lagged that fix. To reset it in the DB: generate a bcrypt hash locally and UPDATE the "user" table in bms-4-n8n-postgres-1.

If Playwright MCP is unavailable, use Option B (manual) below.

Rotation steps

Option A — Playwright agent (preferred)

Spawn a background agent with this prompt (do NOT include the actual password value — the agent reads it from SOPS):

You are rotating BMS4_N8N_API_KEY for the self-hosted n8n instance on bms-4.

IMPORTANT: Never display any secret value in your response. Reference key names only.

1. Read the n8n admin password from SOPS:
   $env:SOPS_AGE_KEY_FILE = "C:\Users\konar\.age\p24-infra-keys.txt"
   Store the value of BMS4_N8N_ADMIN_PASSWORD from secrets/n8n-bms4.env.sops silently in $env:N8N_PASS
   (use Select-String pattern — never print $env:N8N_PASS)
   NOTE: bms4_n8n_db_password is the PostgreSQL password — do NOT use it for n8n UI login.

2. Use Playwright to navigate to https://n8n.bms-4.infra.zintegrowana.online
   - Login with email radieu@gmail.com and the password from step 1
   - Go to Settings → API (left sidebar)
   - If an existing key named "p24-infra-api" exists, click the trash icon to delete it
   - Click "Create an API Key"
   - Set label: p24-infra-api-<YYYY-MM-DD>
   - Copy the new key value into $env:NEW_KEY (never print it)
   - Close the dialog

3. Update secrets/n8n-bms4.env.sops with the new BMS4_N8N_API_KEY value
   Follow the SOPS write pattern from CLAUDE.md (decrypt → edit → re-encrypt → canary).
   Use [System.IO.File]::WriteAllText with UTF8Encoding($false).
   Temp file must be named secrets/n8n-bms4-edit.env.sops

4. Update GH Secret:
   $env:NEW_KEY | gh secret set BMS4_N8N_API_KEY --repo radieu/p24-infra
   $env:NEW_KEY = ""

5. Verify the new key works:
   $r = Invoke-RestMethod "https://n8n.bms-4.infra.zintegrowana.online/api/v1/workflows" `
     -Headers @{ "X-N8N-API-KEY" = $env:NEW_KEY }
   Write-Host "Workflows count: $($r.data.Count)"
   $env:NEW_KEY = ""

6. Commit and open a PR targeting main:
   git add secrets/n8n-bms4.env.sops
   git commit -m "chore: rotate BMS4_N8N_API_KEY (scheduled 90d)"
   Push and open PR via gh pr create.

7. Append to docs/secrets-rotation-log.md:
   | <YYYY-MM-DD HH:MM UTC> | — | BMS4_N8N_API_KEY | scheduled 90d rotation | AI-agent | SOPS + GH Secret |

Option B — Manual (fallback)

Step 1 — Log into n8n
  URL: https://n8n.bms-4.infra.zintegrowana.online
  Email: radieu@gmail.com
  Password: read bms4_n8n_db_password from secrets/n8n-bms4.env.sops (never print value)

Step 2 — Delete old API key
  → Settings (left sidebar) → API
  → Find key named p24-infra-api or similar
  → Click trash icon → Confirm deletion

Step 3 — Create new API key
  → Click "Create an API Key"
  → Label: p24-infra-api-<YYYY-MM-DD>
  → Copy value immediately (shown once only)

Step 4 — Update secrets/n8n-bms4.env.sops (PowerShell, Windows dev workstation)
  $env:SOPS_AGE_KEY_FILE = "C:\Users\konar\.age\p24-infra-keys.txt"
  $plain = sops --decrypt --input-type dotenv --output-type dotenv secrets\n8n-bms4.env.sops
  $tempPath = "$PWD\secrets\n8n-bms4-edit.env.sops"
  # Store new key silently — never print it
  $env:NEW_KEY = "<paste-from-step-3>"
  $updated = $plain -replace "^BMS4_N8N_API_KEY=.*", "BMS4_N8N_API_KEY=$env:NEW_KEY"
  [System.IO.File]::WriteAllText($tempPath, ($updated -join "`n") + "`n",
    [System.Text.UTF8Encoding]::new($false))
  $enc = sops --encrypt --input-type dotenv --output-type dotenv $tempPath
  [System.IO.File]::WriteAllText("$PWD\secrets\n8n-bms4.env.sops",
    ($enc -join "`n") + "`n", [System.Text.UTF8Encoding]::new($false))
  # Canary — verify before commit
  sops --decrypt --input-type dotenv --output-type dotenv secrets\n8n-bms4.env.sops | Out-Null
  if ($LASTEXITCODE -ne 0) { throw "SOPS corrupt — do NOT commit" }
  [System.IO.File]::Delete($tempPath)

Step 5 — Update GH Secret
  $env:NEW_KEY | gh secret set BMS4_N8N_API_KEY --repo radieu/p24-infra
  $env:NEW_KEY = ""

Step 6 — Commit and push
  git add secrets/n8n-bms4.env.sops
  git commit -m "chore: rotate BMS4_N8N_API_KEY (scheduled 90d)"
  git push
  gh pr create --base main --title "chore: rotate BMS4_N8N_API_KEY"

Step 7 — Append to docs/secrets-rotation-log.md
  | <YYYY-MM-DD HH:MM UTC> | — | BMS4_N8N_API_KEY | scheduled 90d rotation | radieu | SOPS + GH Secret |

Verification

# 1. Confirm SOPS updated (key present, check length > 200 as proxy for valid key)
$env:SOPS_AGE_KEY_FILE = "C:\Users\konar\.age\p24-infra-keys.txt"
$line = sops --decrypt --input-type dotenv --output-type dotenv secrets\n8n-bms4.env.sops |
    Select-String "^BMS4_N8N_API_KEY="
$val = $line.ToString().Split('=', 2)[1]
Write-Host "BMS4_N8N_API_KEY length: $($val.Length)"   # expected: >200 chars
 
# 2. Confirm key works against n8n API
$env:K = $val
$r = Invoke-RestMethod "https://n8n.bms-4.infra.zintegrowana.online/api/v1/workflows" `
    -Headers @{ "X-N8N-API-KEY" = $env:K }
Write-Host "Active workflows: $($r.data.Count)"
$env:K = ""; $env:SOPS_AGE_KEY_FILE = ""

Escalation

Symptom	Action
n8n Settings → API page not visible	Upgrade n8n or check if API feature is enabled in instance settings
Login fails	Verify `bms4_n8n_db_password` in SOPS is current; check n8n container logs on bms-4
API call returns 401 after rotation	Wait 30s (cache flush); if still failing, check SOPS was committed and secrets-sync.yml has run
secrets-sync.yml not yet merged	Use the new key directly; service will self-heal on merge + container restart
Key not accepted anywhere	n8n may need a container restart: `ssh root@54.36.123.110 "cd /opt/p24-infra && docker compose restart n8n"`

Prevention

The credential-rotation.yml GH Actions workflow (Monday 06:00 UTC) opens a human-action issue when next_due passes.
n8n does not emit expiry warnings for API keys — only the calendar/GH Actions check triggers rotation.
After rotation, update this playbook’s Last rotated and Next due dates.

Audit Log — Log to infra_operations

After this operation completes, log it to the infra_operations audit table.

Python (Linux server — bms-4, vps-i1, vps-h1, or similar):

import sys
sys.path.insert(0, '/opt/p24-infra')
from scripts.lib.log_op import log_op
 
log_op(
    actor="claude",  # "radieu" for manual human ops, "claude" for agent
    op_type="credential_rotation",
    resource="BMS4_N8N_API_KEY",
    result="success",  # "success" | "failed" | "skipped"
    detail="Scheduled rotation — n8n bms-4 API key regenerated via n8n UI and SOPS updated",
    env="bms-4",
    gh_issue=2730,
)

PowerShell (Windows dev machine):

$env:SUPABASE_URL = (Get-Content "C:\code_2026\p24-infra\.env.local" | Select-String "^SUPABASE_URL=").ToString().Split("=",2)[1].Trim()
$env:SUPABASE_SERVICE_KEY = (Get-Content "C:\code_2026\p24-infra\.env.local" | Select-String "^SUPABASE_SERVICE_KEY=").ToString().Split("=",2)[1].Trim()
python -c "
import os, sys
sys.path.insert(0, 'C:/code_2026/p24-infra')
from scripts.lib.log_op import log_op
log_op('claude', 'credential_rotation', 'BMS4_N8N_API_KEY', 'success', 'Scheduled rotation — n8n bms-4 API key regenerated via n8n UI and SOPS updated', 'bms-4')
"
$env:SUPABASE_URL = ''; $env:SUPABASE_SERVICE_KEY = ''

p24-infra Docs

Explorer

n8n-bms4-api-key-rotation