Secrets Rotation Log — 2026-07-21

Newest first. Appended to by docs/secrets-rotation-log.md conventions (see that file’s header rules).

Date	Secret	Reason	Rotator	Confirmed in sync
2026-07-21 — NOT ROTATED (leak already dead) — DRIFT RE-SYNCED instead	SUPABASE_DB_PASSWORD (secrets/monitoring.env.sops + secrets/n8n-bms4.env.sops; source of truth secrets/et-operational-platform.env.sops)	#4439 (exec of #4110, filed from the #4097 CI-log leak). Verify-first overturned the premise — no rotation performed. Key resolved to 4 SOPS files; brandpilot.env.sops is a different Supabase project (siaarpvchmeqadrltuey) and is legitimately unrelated → out of scope. The 3 copies for project mwkqmgadqnkkihjdeqsi (et-op / monitoring / n8n-bms4) had drifted to 3 distinct values. Direct psql auth test against the live Supavisor pooler (postgres.mwkqmgadqnkkihjdeqsi, via the bms-4 socat proxy 54.36.123.110:15432, exit-code + fingerprints only, no values printed): et-op copy = AUTH OK (the live password); monitoring + n8n-bms4 = password authentication failed. The leaked credential is already dead: the cleartext value in the #4097 run log (29187939637) fingerprints to d2411f1d74d8 — byte-identical to the n8n-bms4 copy (32-char), and that value is now rejected by the pooler. So the DB password was already rotated at Supabase by an earlier actor who updated only the et-op SOPS copy, leaving the other two stale. Rotating again would retire a healthy key for zero security gain → per the issue’s own verify-first gate, rotation declined. The real defect is the drift, and it is actively breaking CI: Apply Supabase Migrations reads /opt/p24-infra/bms-4/.env (written from n8n-bms4.env.sops) and db-maintenance-check.yml reads GH Secret SUPABASE_DB_PASSWORD (synced from monitoring.env.sops per the #4095 allowlist) — both sources stale, both workflows failing on every run (migrations 5/5 failures since 07-06; db-maintenance 5/5 since 07-18). Fix applied: propagated the proven-valid et-op value into both stale files (no new secret minted, nothing rotated at the provider). Per-file write used decrypt→awk(ENVIRON)→re-encrypt into secrets/*-tmp.env.sops (so .sops.yaml path_regex applies all recipients — never --age), with canary decrypt and 6/6 age-recipient count and key-count equality asserted on the temp file before the atomic mv — neither production file was touched until its replacement was proven decryptable. Post-write: all 3 copies now fingerprint-identical (dbf22527c634) and all 3 AUTH OK; per-key fingerprint diff vs origin/main shows SUPABASE_DB_PASSWORD as the only changed key in each file (monitoring 95 keys, n8n-bms4 72 keys, both unchanged) — no collateral drift. Out of scope (untouched): brandpilot.env.sops (different project); every other key in both files; no provider-side password reset; no et-op Vercel redeploy (et-op’s own copy was already correct, so #4438’s redeploy path is not needed for this key).	claude (infra-task worker, infra-task-request-worker.md, issue #4439, queue row 24)	SOPS monitoring.env.sops ✅ canary OK 6/6 recipients, 95 keys intact · n8n-bms4.env.sops ✅ canary OK 6/6 recipients, 72 keys intact · cross-file fingerprints MATCH (3/3) · live pooler auth OK on all 3 copies · only-SUPABASE_DB_PASSWORD-changed verified vs origin/main · PR → main pending review · secrets-sync (→ bms-4 .env + GH Secret) · CI green-check on Apply Supabase Migrations + db-maintenance-check: pending post-merge · old value revoked at provider: N/A — already dead before this task began (verified rejected)
2026-07-21 — SOPS ROTATED + DISTRIBUTED, runtime effect BLOCKED (#4427)	CRON_SECRET (secrets/monitoring.env.sops + secrets/et-operational-platform.env.sops)	bypass-batch 1/9 scheduled rotation (#4424, tracker #3351). Tier 1 self-generated random token — no provider dashboard, no external party to revoke against, so secret-manager holds full authority and the rotation is autonomous. Verify-first performed before any write (key names + fingerprints only, no values): CRON_SECRET resolved to exactly 2 SOPS files across all 26 secrets/*.env.sops (monitoring + et-operational-platform — matches the topology recorded in the 2026-07-13 #4057 entry); both copies were 64-char, non-placeholder, and fingerprint-identical to each other, i.e. already correctly in sync from PR #4064 (merged 2026-07-13). Consumer-contract check → clear to rotate: CRON_SECRET is a self-compared bearer token — Vercel Cron sends Authorization: Bearer $CRON_SECRET to et-operational-platform’s own /api/cron/* routes, which compare against the same project env var; both sides are updated from the same SOPS key by the same sync run, so there is no counterparty holding a pinned value. Zero runtime references in scripts/, monitoring/, infra-src/ (the monitoring copy is a source-of-truth mirror, not a live caller) and zero secrets.CRON_SECRET reads in .github/workflows/. Generated fresh 64-char hex (openssl rand -hex 32); applied per-file via decrypt→awk-replace→re-encrypt into a *-tmp.env.sops inside secrets/ (so .sops.yaml path_regex applies all recipients — never --age), with a canary decrypt and a 6/6 age-recipient count on the temp file before the atomic mv, so neither production file was touched until its replacement was proven decryptable. Post-write verification (fingerprints only): new value matches across both files; per-key fingerprint diff vs origin/main shows CRON_SECRET as the only changed key in each file, with keysets intact (monitoring 96 keys, et-op 27 keys) — no collateral drift. Distribution-surface gap found and fixed in the same PR: a GH Actions Secret named CRON_SECRET has existed in radieu/p24-infra since 2026-07-05 but was never in the sync_keys_from_file /tmp/monitoring.env allowlist in secrets-sync.yml, so both the 2026-07-13 (#4057) and this rotation would have left it stale. No workflow currently reads secrets.CRON_SECRET, but scripts/rotate-credentials.py writes it (gh_secret_set('CRON_SECRET', …)), so the store is part of the key’s distribution surface. Same drift class as PDF_SERVICE_API_KEY (#4090), SUPABASE_DB_PASSWORD (#4095), OPENAI_ADMIN_KEY/EMAIL_SENDER_API_KEY (#4093), BMS4_N8N_API_KEY (#3641). The allowlist fix was prepared and YAML-validated but could not ship in this PR — the infra-task worker’s GitHub token lacks the workflow OAuth scope, so pushing any .github/workflows/** change is rejected. Filed separately as #4425; this PR is SOPS + docs only. Consequence until #4425 merges: GH Secret CRON_SECRET stays at its 2026-07-05 value (harmless today — no workflow reads it). No container --force-recreate required: the key is an HTTP auth string read per-request from Vercel’s env, not a baked container env; no W3/W4 surface is touched. Out of scope (untouched): every other key in both files; bypass-batch items 2/9–9/9. POST-MERGE VERIFICATION — the rotation is NOT live (#4427). secrets-sync.yml run 29807737769 succeeded on all 12 jobs; Vercel logged Replaced: CRON_SECRET (1 old entries) on both et-op projects (prj_ziLl911…, prj_lli8aw…, 27 vars each) and vps-i1 received the new monitoring/.env with containers recreated. But Vercel bakes env vars into a deployment at build time, and no sync step triggers a redeploy. The newest et-op production deployment is dpl_H14maGjibputj8Bca8uDJK2kqwUZ from 2026-07-05 — it predates both this rotation and #4057’s. Zero-side-effect probe against POST /api/cron/task-metrics (validates auth before body, so a valid secret + empty body ⇒ 400 with no DB write): invalid control ⇒ 401, new secret ⇒ 401, previous (#4057, 07-13) secret ⇒ 401. Both rejected ⇒ the live baked value is the pre-07-13 one. Consequence: #4057’s rotation was also inert — the credential it meant to retire on 2026-07-13 is still the live secret, and this generalises to all 27 et-op keys (incl. SUPABASE_SERVICE_ROLE_KEY, GITHUB_TOKEN, QUEUE_API_KEY). Redeploy deliberately NOT performed by this worker: prod is pinned at a 07-05 build while main advanced ~2 weeks, so “redeploy latest” would ship unreviewed commits and “rebuild same commit” still needs a human to confirm the pin isn’t a deliberate freeze. Escalated as #4427.	claude (infra-task worker, infra-task-request-worker.md, issue #4424)	SOPS: monitoring.env.sops ✅ canary OK 6/6 recipients · et-operational-platform.env.sops ✅ canary OK 6/6 recipients · cross-file fingerprints MATCH · only-CRON_SECRET-changed verified vs origin/main · PR #4426 MERGED · secrets-sync 29807737769 ✅ 12/12 jobs · Vercel project env ✅ both projects · vps-i1 /opt/p24-infra/monitoring/.env ✅ · GH Secret ❌ stale — not in allowlist (#4425) · Live runtime value ❌ NOT rotated — blocked on et-op redeploy (#4427) · old value revoked at provider: N/A — self-generated internal token, no provider
2026-07-21 00:00 UTC	PINBOX24_W4_WASABI_ACCESS_KEY + PINBOX24_W4_WASABI_SECRET_KEY (secrets/et-operational-platform.env.sops) — read-only Wasabi IAM key for user pinbox24-et-op-reader (scope s3:GetObject+s3:ListBucket on pinbox24/offices/*; consumed by et-op /zarzad/pinbox PDF preview via Vercel env)	exposure (#4444, tracker #2714 / P0 #2824) — key was exposed in the 2026-07-05 Session-31 plaintext dump and marked PENDING_HUMAN, never rotated (same value still live). Gate to rotate cleared by 4438 (secrets-sync now redeploys the live et-op commit, so the new value bakes into prod instead of revoking-while-stale). Tier 1 autonomous per wasabi-key-rotation.md + access matrix. Verify-first: confirmed current key Active + S3-functional (eu-central-1, pinbox24/offices listable) and WASABI_ADMIN valid before any mutation; current key last_used=NEVER (near-zero blast radius). Minted fresh access key via WASABI_ADMIN boto3 CreateAccessKey(pinbox24-et-op-reader), verified new key against s3://pinbox24/offices/ (eu-central-1) BEFORE writing SOPS. SOPS edited in place (temp→encrypt→canary→replace; 6/6 age recipients preserved, key-name set unchanged). Old key deactivated first (reversible) → verified rejected (InvalidAccessKeyId) → new key re-checked OK → old key deleted; user now has exactly one Active key (the new one). No secret values printed anywhere (key NAMES + masked IDs only).	claude (secret-manager/infra-task worker, bms-4, queue row 3318)	SOPS secrets/et-operational-platform.env.sops ✅ canary OK, 6/6 recipients · PR #4451 → main merged (6e99221) ✅ · secrets-sync.yml run 29859975667 sync-et-operational-platform ✅ (Vercel env update + same-commit redeploy) · new key S3-verified pre-write + post-deploy ✅ · old key …YBBY deactivated → InvalidAccessKeyId → deleted ✅ · final: 1 Active key (…KFKH) on pinbox24-et-op-reader