Plan 4556 — Role- and repo-scoped SOPS secret access

Issue: #4556 Status: Draft — awaiting review Author: worker agent (dev-issue, vps-i1) Created: 2026-07-29 Role note: This document is a design artifact only. Every live secrets/*.env.sops write described here MUST be executed by /role-secret-manager (or filed as an infra-task for a sys-admin/secret-manager session). A generic worker never re-encrypts a SOPS file.


0. TL;DR

Today, one age keypair per machine grants that machine decrypt access to ~24 of the 26 secrets/*.env.sops files. bms-4 — the most autonomous, least-reviewed code-execution host in the fleet — can therefore decrypt MongoDB rs0 admin, Vercel, Wasabi, Supabase service-role, GitHub App signing keys, and every application repo’s secrets, on any job, regardless of what that job is for.

This plan replaces per-machine keys with one age keypair per SOPS-file scope, and delivers each scope key to a worker ephemerally at dispatch time based on the job’s already tracked role + repo_context. The blast radius of a compromised worker job shrinks from “the entire estate” to “the one scope that job legitimately needs”.

What this plan does NOT do: it does not reduce what a compromised developer workstation or a compromised GitHub Actions environment can reach. See §11 (Residual risk).


1. Current state — verified, not assumed

All facts below were verified against this branch on 2026-07-29 by inspecting file contents and key names only. No secret value was decrypted, printed, or logged.

1.1 .sops.yaml — three creation_rules

RuleRecipientsFiles matched
secrets/administration.env.sopsdeveloper key only1
secrets/pinbox24-radieu-w4-auth.env.sopsdeveloper key only1
secrets/.*\.env\.sops (catch-all)6: developer, AGE_KEY_GHA (CI), bms-4 runner, vps-i1 runner, dev-laptop, vps-h1 runner25

secrets/ currently holds 27 *.env.sops files. Note this includes the dotfile secrets/.canary.env.sops, which a plain ls secrets/ hides but which does match the catch-all regex (^|[/\\])secrets[/\\].*\.env\.sops$ — so it is one of the 25. Any tooling that enumerates scopes must glob dotfiles too, or it will silently skip it.

Two files carry .sops.yaml comments describing them as “admin-only” — bms-servers.env.sops and gcp.env.sops — but both fall under the flat 6-recipient rule. They are deployment-excluded (dropped by secrets-sync.yml’s path filter), not recipient-isolated. Any host’s universal key decrypts them directly. This is the single clearest illustration of the gap this plan closes: the policy layer says admin-only, the crypto layer says everyone.

1.2 Concrete blast radius — secrets/n8n-bms4.env.sops

~68 keys, routinely decrypted by bms-4. Key names present include:

MONGODB_RS0_ADMIN_PASSWORD · MONGODB_RS0_PROMETHEUS_PASSWORD · GITHUB_APP_ID · GITHUB_APP_PRIVATE_KEY_B64 · GITHUB_APP_INSTALLATION_ID · VERCEL_TOKEN · SUPABASE_SERVICE_ROLE_KEY · SUPABASE_DB_PASSWORD · WASABI_ACCESS_KEY · WASABI_SECRET_KEY · RADEK_BRAND_LINKEDIN_ACCESS_TOKEN · RADEK_BRAND_MAILERLITE_API_KEY

— alongside the genuine n8n/Redis/Atrax operational secrets that justify the file’s existence. The RADEK_BRAND_* pair belongs to a different repo entirely.

MONGODB_RS0_ADMIN_PASSWORD is confirmed still present in this file on both this branch and origin/main, matching the issue’s premise.

1.3 One keypair per machine

ansible/roles/claude-env-sync/tasks/main.yml generates exactly one keypair per host at /home/claude-runner/.age/p24-infra-keys.txt (age-keygen -o <path>, when: not age_key_stat.stat.exists), and prints only the public key in a debug task. There is no existing mechanism to scope more narrowly than “this machine”.

1.4 Dispatch already knows role + repo — but SOPS access ignores both

infra-src/meta-dispatcher/src/classify.ts classifies every queued task into { role, repo_context, server_preference }, with DEV_REPOS = { et-operational-platform, art-agency, brandpilot, radekkonarski-personal-brand }. scripts/queue-dispatcher-loop.py carries those fields through to scripts/spawn-worker.sh, which receives ROLE ($7), REPO_CONTEXT ($8) and JOB_PROFILE ($9).

And yet, in scripts/spawn-worker.sh:

export SOPS_AGE_KEY_FILE="${HOME}/.age/p24-infra-keys.txt"   # one key, every job

The SOPS_FILE case statement immediately below maps repo → SOPS file, but it only exports a path string. Nothing enforces it. A dev-issue job for brandpilot is handed the universal key and can sops -d any of the 25 catch-all files.

Evidence that the mapping is unenforced: two of its seven entries point at files that do not exist (verified 2026-07-29):

Mapped repoSOPS_FILE targetExists?
radekkonarski-personal-brandsecrets/radekkonarski-brand.env.sopsMISSING
amazon-kdp-tangosecrets/kdp-tango.env.sopsMISSING
brandpilot, art-agency, p24-infra, et-operational-platform, whatsup-android-chat-pullerOK

Because nothing validates the path, these fail silently — the job proceeds with the universal key and full catch-all access, which is strictly worse than failing closed. This is a pre-existing bug, not caused by this plan; it should be filed as its own follow-up issue rather than fixed in this doc-only PR. It is recorded here because Phase 4 replaces this exact code path, and the scoped replacement must fail loud where this one fails silent (§7.3).

1.5 The role axis is CI-mediated — and currently bypassable

Five secrets/role-*.env.sops files exist (sys-admin, secret-manager, dev-coder, dev-reviewer, dev-tester). secrets-sync.yml decrypts them server-side with AGE_KEY_GHA and scps the plaintext to /opt/p24-infra/{server}/role-{role}.env; spawn-worker.sh then sources the file matching the job’s JOB_TYPE.

That path is sound. The hole is that all five files also sit under the flat 6-recipient rule, so any host’s universal key can decrypt them directly, bypassing the CI-mediated delivery entirely. Fixing this needs no new keypairs — only recipient removal.

1.6 The ephemeral-delivery precedent already exists

spawn-worker.sh (l.458–487) materializes ROLE_GIT_SIGNING_KEY from the role env into a 600-mode file under ${XDG_DATA_HOME:-$HOME/.local/share}/p24-worker/signing-keys, runs a usability preflight, hands ownership to the detached worker, and sweeps orphans older than 12h. This is the shape to reuse — including its scar tissue (§7.3).


2. Coordination — what already exists, and what this plan defers to

Searched and confirmed on 2026-07-29:

RefStateRelationship
#2732OPEN”Remove duplicate MONGODB_RS0_ keys from n8n-bms4.env.sops (moved to mongodb-bms.env.sops)”. This is the pre-existing execution issue for what the #4556 body calls “Phase 0”.
#3178OPENmonitoring.env.sops role-based reorganisation, plan already written at docs/plans/monitoring-env-sops-reorganisation.md. The content axis.
#2835OPEN, P0Full SOPS rebuild — recreate all files with fresh keys. Hard sequencing constraint.
#2416CLOSEDPer-service credential isolation plan delivered, execution tracked via #2732 / #2728.
#4068CLOSED, shippedGitHub App installation tokens — the validated ephemeral-credential precedent.

2.1 Phase 0 is not re-derived here

The #4556 body asks to “execute that specific move as Phase 0”. #2732 already owns that work and is open. Re-deriving it in this plan would create exactly the duplicate the issue warns against. This plan therefore:

  • Declares #2732 a hard prerequisite of Phase 5, and does not restate its steps.
  • Contributes one thing #2732 does not currently assert: that after the move, the recipient set of mongodb-bms.env.sops must be narrowed too — otherwise the credential has moved file but not shrunk blast radius, since both files sit under the same flat rule.

2.2 Sequencing against #2835 (the decision that gates everything)

#2835 (P0) plans to recreate every secrets/*.env.sops with fresh values. A recipient migration and a full value rebuild both rewrite every file. Running them concurrently risks a lost-update collision on files touched by both, with the failure mode being an undecryptable production secret.

Recommendation: fold the recipient change into #2835 rather than racing it. #2835 already recreates each file from scratch; writing it out under a narrowed creation_rule costs one extra .sops.yaml edit per file and zero extra sops updatekeys runs. Doing them separately means encrypting every file twice.

This is the plan’s single biggest open decision and needs a human call before Phase 5 begins. Phases 1–4 (key generation, tooling, verification harness) are **independent of 2835 and can proceed immediately in either case — they add capability without changing any recipient.


3. Threat model — be precise about what is being defended

In scope. A worker job on a dispatch host (primarily bms-4) executes attacker-influenced input — a crafted issue body, a malicious dependency pulled during implementation, a prompt-injected file in a cloned repo — and attempts to read credentials beyond its task.

Why bms-4 specifically. It is the primary dispatch node, runs the least-reviewed code paths (headless Claude executing arbitrary issue instructions), and holds vps_root_key (root SSH to the whole fleet — a standing P1 concern in docs/priorities.md).

Out of scope, by design.

  • Compromise of the developer workstation (holds the break-glass key).
  • Compromise of GitHub Actions (holds AGE_KEY_GHA).
  • OS/container-level sandboxing of worker jobs — explicitly out of scope per the issue; tracked as future hardening (§12).
  • An attacker with root on a dispatch host during a job that legitimately holds a scope key. Scoping bounds what one job reaches; it does not defend a fully-owned host.

What “shrink the blast radius” concretely means here: a role=dev-coder, repo=brandpilot job moves from 24 files decryptable to 2 (brandpilot.env.sops plus its role credential set, the latter delivered as plaintext by CI, not decrypted on-host).


4. Target architecture

4.1 Unifying principle

One age keypair per SOPS-file scope, not per host.

The file layout already encodes the right granularity. Reuse it instead of inventing a new taxonomy.

4.2 Recipient formula

For each secrets/<scope>.env.sops, the creation_rule recipients become exactly:

  1. Developer key — human break-glass identity. Unchanged from today. Present on every rule.
  2. AGE_KEY_GHAonly if secrets-sync.yml actually deploys that file. Files already excluded by its path filter (administration, bms-servers, ovh-api, gcp, whatsup, rotate-secret, pinbox24-radieu-w4-auth) do not get CI as a recipient. This alone is a meaningful reduction: CI currently holds keys to files it never deploys.
  3. That scope’s own dedicated keypair — private half installed only on hosts that actually run jobs needing it (per classify.ts routing: most scopes only ever need bms-4 today).

The four universal per-host keys are removed only after the scope key is installed and verified (§7.2 dual-key window).

4.3 No standing master key for automation

Deliberately no “secret-manager master key” as a permanent recipient on every file. That would relocate the blast-radius problem onto one new identity rather than solve it. secret-manager access splits into two tiers:

TierWhoHow access is obtained
Routinejob_type=secret-manager queue rows naming a specific scope (“rotate N8N_ENCRYPTION_KEY”, “add KEY_X to brandpilot.env.sops”)dispatched workerResolves scope from the request, gets the same ephemeral scope key any dev-coder job gets, plus re-encrypt capability for that one scope.
Cross-cutting — new scope creation, fleet-wide sops updatekeys, executing this migrationhuman-operated /role-secret-manager session, or CIUses the developer key or AGE_KEY_GHA. No new identity.

Post-migration, no automated or dispatched identity holds standing at-rest access to every file. Only the developer key and CI do — exactly as today, and both are outside this plan’s threat model.

4.4 Role axis: recipient removal only

Role credentials are delivered exclusively via the CI-mediated pre-decrypt-and-deploy path (§1.5). No live sops -d of a role-*.env.sops file ever happens on a host. Therefore the five role files need no scope keypair — only removal of the four universal host keys, leaving developer + AGE_KEY_GHA.

This is the cheapest, lowest-risk, highest-value change in the whole plan. It should ship first (Phase 3) and independently.

4.5 Ephemeral delivery (repo axis)

At dispatch, spawn-worker.sh resolves (role, repo_context, job_type) → scope, materializes that scope’s private key to a 600-mode file under the existing ${XDG_DATA_HOME:-$HOME/.local/share}/p24-worker/ tree, exports SOPS_AGE_KEY_FILE to point at it, hands ownership to the detached worker, and the worker shreds it in its own cleanup. Orphan sweep mirrors the existing 12h signing-key sweep.

The scope private keys themselves are stored at rest in a new secrets/scope-keys.env.sops — recipients developer + AGE_KEY_GHA only — and deployed by secrets-sync.yml to each dispatch host as /opt/p24-infra/{server}/scope-keys.env, mode 600, owned by claude-runner. This reuses the exact delivery path already proven for role-*.env, and means the host no longer holds a broad age key at all — it holds a bundle from which the spawner extracts one entry per job.

Honest limitation, stated plainly. A host holding the scope-key bundle can read any key in it. This design bounds what a job reaches, not what a host reaches. Reducing the latter requires per-job OS isolation (§12) or a network key broker (§12). This is a real and deliberate limitation of the ephemeral-materialization approach; it should not be described as full isolation. The gain is genuine but bounded: it removes at-rest, always-on, whole-estate access from the process that executes untrusted input, and it makes every scope access an explicit, loggable event (§9).

4.6 New scope: secrets/platform-integrations.env.sops

Extract from n8n-bms4.env.sops the keys that are platform-wide integrations rather than n8n operational secrets: GITHUB_APP_ID, GITHUB_APP_PRIVATE_KEY_B64, GITHUB_APP_INSTALLATION_ID, VERCEL_TOKEN, SUPABASE_SERVICE_ROLE_KEY, SUPABASE_ANON_KEY, SUPABASE_URL, WASABI_ACCESS_KEY, WASABI_SECRET_KEY. MONGODB_RS0_* is #2732’s business, not this plan’s.

RADEK_BRAND_* — destination must be decided, not assumed. These keys (RADEK_BRAND_LINKEDIN_ACCESS_TOKEN, RADEK_BRAND_MAILERLITE_API_KEY) belong to radekkonarski-personal-brand, whose SOPS file lives in that repo, not here — secrets/radekkonarski-brand.env.sops does not exist in p24-infra/secrets/ (verified 2026-07-29). So there is no in-repo destination to move them to today. Two options, needing a human call in Phase 5:

  • (a) Relocate them to the radekkonarski-personal-brand repo’s own SOPS file and delete them here. Cleanest, but a cross-repo secret move — must go through secret-manager and verify no p24-infra consumer reads them first (grep -rl RADEK_BRAND_ before removal).
  • (b) Create secrets/radekkonarski-brand.env.sops in p24-infra as a proper scope, which would also make spawn-worker.sh’s existing (currently dead — §1.4) mapping resolve.

Do not silently drop them into platform-integrations.env.sops: they are neither platform-wide nor n8n-operational, and folding them in would re-create in a new file exactly the “unrelated repo’s secrets riding along” problem this plan exists to remove.

Leave secrets/n8n-bms4-gh.env.sops alone (GH_TOKEN, GITHUB_PAT_ALL_WRITES, GITHUB_PAT_P24_INFRA) — already a separate, isolated concern, out of this pilot’s scope.


5. ⚠️ Verified breakage risk — scripts/gh-app-token

This is the finding most likely to break production if Phase 5 is executed naively.

scripts/gh-app-token (208 lines, Python) does not read from SOPS. It reads GITHUB_APP_ID / GITHUB_APP_PRIVATE_KEY_B64 / GITHUB_APP_INSTALLATION_ID from $GH_APP_ENV, else from the first of:

ENV_CANDIDATES = [
    "/opt/p24-infra/bms-4/.env",
    "/opt/p24-infra/monitoring/.env",
]

that actually carries GITHUB_APP_PRIVATE_KEY_B64 — i.e. from deployed plaintext env files. Consumers found: bms-4/git-credential-gh-token.sh, scripts/setup-git-credential-helper.sh, scripts/spawn-worker.sh, secrets-sync.yml.

Two consequences:

  1. Moving the GITHUB_APP_* keys into platform-integrations.env.sops is not enough. secrets-sync.yml must be updated so those keys still land in the deployed env file that ENV_CANDIDATES reads — or ENV_CANDIDATES / GH_APP_ENV must be updated to the new deployed path. If neither happens, token minting breaks fleet-wide, taking git push, the credential helper, and worker GitHub access with it.

  2. A pre-existing discrepancy, flagged for verification. scripts/gh-app-token’s docstring states the App keys are deployed to /opt/p24-infra/monitoring/.env on vps-i1. But secrets/monitoring.env.sops contains zero GITHUB_APP_* keys, and no workflow, template, or role-*.env.sops file references them — they exist only in n8n-bms4.env.sops. So on vps-i1 neither candidate should carry the key. Scope of this check: SOPS sources and repo files only. The live server file was not inspected from this worker. A sys-admin session should confirm whether gh-app-token currently works on vps-i1 before Phase 5 assumes either answer.

Phase 5 gate: scripts/gh-app-token --check must pass on every dispatch host both before and after the move, with the before-result recorded in the migration log.


6. Phases

Phases 1–4 add capability and change no recipients — they are safe to run regardless of the #2835 decision. Phases 5–8 change recipients and are gated on §2.2.

PhaseWorkRoleGated on
0MongoDB admin credential movesecret-manager**Deferred to 2732
1Generate scope keypairs; create secrets/scope-keys.env.sops; extend claude-env-sync to install the bundle; register new scopes in dev_r_servicessecret-manager + dev-coder
2scripts/verify-sops-scope-isolation — pass/fail matrix per (key, file), values never printeddev-coder
3Role-file recipient narrowing — drop the 4 universal host keys from the 5 role-*.env.sops rulessecret-managerPhase 2
4spawn-worker.sh scope resolution + materialization + preflight + shred; queue-dispatcher-loop.py passes scope throughdev-coderPhase 1
5Pilotbrandpilot.env.sops narrowed end-to-end; create platform-integrations.env.sopssecret-manager§2.2, #2732, §5 gate
6Pilot verification — positive + negative decrypt, live job runsys-adminPhase 5
7Roll out to remaining scopes, one file per PR, visible checklistsecret-managerPhase 6
8Migration log; wire Phase 2 script into CI on every .sops.yaml changedev-coderPhase 7
9Decrypt-event audit logging + staleness detectiondev-coderPhase 8

6.1 Phase 3 detail — the quick win

For each of the five secrets/role-*.env.sops, add an explicit creation_rule above the catch-all with recipients developer + AGE_KEY_GHA, then sops updatekeys --yes that file.

Verification: on bms-4, sops -d secrets/role-dev-coder.env.sops must fail with “no matching recipient” (exit code + stderr text only — never the decrypted body). secrets-sync.yml must still deploy all five successfully, since CI retains its recipient.

Rollback: re-add the host keys to the rule, sops updatekeys, push. One commit, one minute.

6.2 Phase 5 detail — pilot on brandpilot.env.sops

Chosen because it is a dev repo (exercises the repo_context axis end-to-end), it is deployed by CI (exercises the AGE_KEY_GHA retention path), and it is low-traffic — a mistake does not take production monitoring or MongoDB down.

Ordering within the phase is non-negotiable:

  1. Add the brandpilot scope key as an additional recipient. sops updatekeys. Canary.
  2. Verify a real repo=brandpilot job decrypts via the scope key.
  3. Only then remove the four universal host keys. sops updatekeys. Canary.
  4. Verify again, positive and negative.

Steps 1 and 3 are separate commits, separate PRs, separate reviews. Never one batch edit.

Rollback (applies to every recipient-removal step, Phases 5 and 7 alike). The removal commit is always revertible in isolation, because the addition of the scope key landed in an earlier, separate commit:

  1. Re-add the removed recipients to that file’s rule in .sops.yaml (or git revert the removal commit — it touches only .sops.yaml plus the re-encrypted file).
  2. sops updatekeys --yes secrets/<file>.env.sops.
  3. Canary: sops -d the file, assert exit code 0 only — never print the output.
  4. Push. Distribution follows the normal secrets-sync.yml path.

The developer key is never removed from any rule at any point in this migration, so break-glass recovery from the workstation is always available even if a rollback is itself botched. If a file is discovered undecryptable on a host mid-job, the immediate mitigation is this rollback — not a re-key — because the plaintext values are unchanged throughout; only recipients move.


7. Risks and mitigations

7.1 Lockout during transition

Risk: a recipient removed before its scope key is installed → file undecryptable on that host, mid-job.

Mitigation: mandatory dual-key window per file (§6.2). Every removal is its own reviewed PR. The developer key is never removed from any rule, so break-glass recovery is always available from the workstation.

A file not yet migrated behaves exactly as today — strictly no worse. This must be stated in the migration log so partial rollout is never mistaken for partial breakage.

7.2 Migration stall after the pilot

Risk: Phase 6 succeeds, Phase 7 never completes, most of the estate stays on the broad model while the pilot PR makes it look done.

Mitigation: Phase 7 is a visible checklist — one task-list line per file on the tracking issue, one PR per file. The Phase 2 verify script prints a per-file migrated/not-migrated column so incompleteness is machine-visible, not a memory item.

7.3 Ephemeral-delivery fragility — inherited, not hypothetical

The ROLE_GIT_SIGNING_KEY pattern being reused has already caused two production incidents:

  • #3385 — the spawner’s EXIT trap deleted the key before the detached worker read it. Applied to age keys, the equivalent is a worker’s sops -d failing mid-job because its scope key was already shredded.
  • #3540 — a malformed / passphrase-encrypted key silently broke signing with no clear error. The age equivalent is worse: a corrupt scope key degrades to “no secrets”, which a worker may misread as “this job has no secrets” and proceed with wrong behaviour rather than failing.

Mitigation:

  • Copy the ownership-handoff discipline exactly: no EXIT trap, narrow TERM/INT trap cleared after successful spawn, persistent XDG dir (not /tmp, which is subject to systemd-tmpfiles GC), 12h orphan sweep.
  • Add an age preflight analogous to the existing ssh-keygen -y -P '' check (search spawn-worker.sh for that literal string rather than a line number — line numbers drift). Validate the materialized key by attempting a decrypt of a tiny known-good canary file for that scope, asserting exit code only.
  • Fail loud, never silent. If the scope key is missing or invalid, the spawner must refuse to start the job and mark the queue row failed with a distinct failure_reason — never fall through to “running without secrets”. The existing SOPS_FILE fallback already logs [WARN] … proceeding without secrets; that permissive behaviour must not be carried into the scoped path.

7.4 Re-widening regression

Risk: a rule silently broadens after the migration. Two distinct routes, both real: a later unrelated .sops.yaml edit that re-widens a recipient list; or a path-string mismatch that causes a narrower file’s content to match the broad catch-all rule instead — which is exactly what #3275 was (sops-set.ps1 temp-file naming caused administration.env.sops content to be re-encrypted under the wrong, broader rule). More narrow rules means more surface for that second route. A one-time Phase 6 test proves isolation then, not later.

Mitigation: Phase 8 wires scripts/verify-sops-scope-isolation into CI as a required check on every .sops.yaml change. Continuous enforcement, not a point-in-time fact.

7.5 Key sprawl / zombie scope keys

Risk: ~6 keys becomes ~20–30. Two failure modes: (a) a key generated in Phase 1 but never wired into the spawner lookup — a dead recipient sitting on a file forever; (b) a retired repo’s scope key never revoked.

Mitigation: Phase 2’s script cross-checks every .sops.yaml recipient against the spawner’s lookup table and flags orphans. Phase 9’s audit logging doubles as staleness detection — flag any scope key with zero decrypt events over N months for review.

7.6 Collision with #2835

Covered in §2.2 — the gating decision.


8. Secret-handling rules for the implementation itself

The migration must not itself become an exposure vector. Non-negotiable, every phase:

  • Key generation (Phase 1): age-keygen -o <path> writes private keys directly to disk. Only the public key may be echoed, logged, or reported (age public keys are non-secret, like an SSH pubkey). No task may pipe a private key file through a step that could echo it.
  • Canary / decrypt verification (Phases 0, 2, 5, 6, 7, 8): assert on exit code and stderr text only (“no matching recipient” vs success). Never sops -d | Out-String, never cat/Read decrypted plaintext, never pipe decrypted content into a log, GH issue, or PR comment.
  • Negative-decrypt tests (Phase 6): capture exit code / stderr only. If the attempt unexpectedly succeeds — the exact bug the test exists to catch — the failure report must say UNEXPECTED SUCCESS — scope leak (file: X, key-id: Y) and still not print the decrypted content.
  • Verify script output (Phase 2): a matrix of pass/fail marks per (key, file) pair. Never a decrypted KEY=value line.
  • Migration log (Phase 8): file names, dates, which recipients were added/removed, who performed the step. Never key values.
  • PR / issue / commit text throughout: reference credentials by key NAME only (MONGODB_RS0_ADMIN_PASSWORD, VERCEL_TOKEN). Never a value, not even truncated.
  • Windows-side edits: [System.IO.File]::WriteAllText($path, $content, [System.Text.UTF8Encoding]::new($false)) only — never >, Out-File, or Set-Content (CRLF/BOM corruption). Canary-decrypt before every git add. Temp files inside secrets/ MUST be named *-tmp.env.sops — SOPS matches creation_rules on the input file path string, so a temp file named otherwise gets encrypted under the wrong rule. See docs/playbooks/sops-edit-operations.md and docs/playbooks/sops-windows-crlf.md.
  • If a secret value ever appears in a log, terminal, commit, or chat output: stop, do not continue the migration step, follow docs/playbooks/static-api-key-incident-rotation.md before resuming.

9. Verification

Phase 2 script — scripts/verify-sops-scope-isolation. For each (scope key, SOPS file) pair, assert decrypt succeeds where the matrix says it should and fails where it should not. Output is a pass/fail matrix only. Exit non-zero on any deviation. Also flags (a) recipients present in .sops.yaml but absent from the spawner lookup, (b) the converse.

Per-file cutover checks (Phases 5–7). Before removal: scope key decrypts (exit 0). After removal: each removed host key fails (non-zero + “no matching recipient”); scope key still succeeds; secrets-sync.yml still deploys the file if it is a deployed file; a real dispatched job of the matching (role, repo) completes.

Continuous (Phase 8). The Phase 2 script runs in CI on every .sops.yaml change.

Phase 5 additional gate. scripts/gh-app-token --check passes on every dispatch host before and after, per §5.


10. Compliance

Per CLAUDE.md §Compliance, each new credential-bearing element is registered in Supabase dev_r_services with compliance_workbook='yes' and a workbook_url, as part of the phase that creates it — not afterwards:

  • secrets/platform-integrations.env.sops (Phase 5)
  • secrets/scope-keys.env.sops (Phase 1)
  • each per-scope keypair (Phase 1)

10.1 Error Notification Standard — mandatory for the new automations

Per CLAUDE.md §Error Notification Standard, every script, GH Action, cron job, and automation must, on error: (1) send a Discord embed via P24_DISCORD_INFRA_SCRIPTS_ERRORS_WEBHOOK_URL, and (2) create a GH Issue in radieu/p24-infra labelled bug. This plan introduces three such automations, and each must ship with that error path in the same PR that creates it:

AutomationPhaseError path required
scripts/verify-sops-scope-isolation2Discord + GH issue on any isolation-assertion failure
CI check on .sops.yaml changes8Discord + GH issue on check failure
Decrypt-event audit logging + staleness detection9Discord + GH issue on ingestion failure and on a detected scope leak

Constraint that overrides the convenience of a verbose alert: the error payload carries file names, key NAMES, and age key IDs only — never a decrypted value, never a partial one. A scope-leak alert reports which (key, file) pair unexpectedly decrypted, not what it decrypted. Otherwise the notification path becomes precisely the exfiltration channel §8 exists to prevent, and Discord + a public-ish GH issue are worse places to leak than a local log. Follow the existing pattern in scripts/backup-git-mirror.sh / scripts/alert-ingest.py.

Ops documentation lands in docs/playbooks/ alongside Phase 8’s migration log. docs/playbooks/secret-repo-access-matrix.md — the policy layer this plan adds crypto enforcement beneath — is updated in Phase 7 to record, per repo, which scope key backs the policy row. docs/playbooks/secret-rotation-access-matrix.md and docs/playbooks/credential-rotation-180d.md gain the new scopes’ rotation entries.


11. Residual risk — what this plan does NOT close

Stated plainly so this is not mistaken for a complete fix:

  1. The developer’s age key remains broadly scoped, by design. It is the break-glass identity every scope still trusts. A compromised developer workstation reaches everything — before and after this migration, unchanged.
  2. AGE_KEY_GHA remains broadly scoped across all CI-deployed files. A compromised GitHub Actions environment reaches every deployed secret. Narrowed slightly by §4.2 rule 2 (CI drops off files it never deploys), but not meaningfully reduced.
  3. A dispatch host holding the scope-key bundle can read any key in it (§4.5). Job-level scoping is bounded by host-level trust.
  4. A fully-compromised host during a legitimate job still reaches that job’s scope.
  5. This does not fix the vps_root_key co-location concern on bms-4 (standing P1 in docs/priorities.md) — a separate credential surface.

What it does close: at-rest, always-on, whole-estate decrypt access from the process that executes the least-reviewed, most autonomous code in the fleet. That is the actual threat model driving the issue, and it is a real reduction — not a complete one.


12. Out of scope / future hardening

  • Full OS/container-level per-job sandboxing — explicitly out of scope per the issue. File as a separate follow-up; it is the only thing that closes §11 item 3.
  • Network key broker — a service that hands a job a decrypted subset rather than a key, removing the bundle from the host entirely. Larger change; the natural successor if §11.3 proves unacceptable.
  • Narrowing AGE_KEY_GHA via per-workflow OIDC-scoped keys.

13. Open decisions for the reviewer

  1. §2.2 — fold into #2835 or run separately? The plan’s biggest sequencing question. Recommendation: fold in. Needs a human call before Phase 5.
  2. §4.5 — is host-level bundle access acceptable as the stopping point for this iteration, with per-job sandboxing deferred?
  3. §5 — does gh-app-token currently work on vps-i1? Needs a sys-admin check before Phase 5 proceeds on either assumption.
  4. §6.1 — ship Phase 3 (role-file narrowing) immediately and independently? It is low risk, high value, and unblocked by everything above. Recommendation: yes.
  5. §4.6 — where do RADEK_BRAND_* go? Option (a) cross-repo relocation, or option (b) create the scope here. Needed before Phase 5 touches n8n-bms4.env.sops.

14. Follow-up issues to file (not fixed by this PR)

WhatWhy separate
Two dead SOPS_FILE mappings in scripts/spawn-worker.sh (§1.4)Pre-existing bug, unrelated to this design; needs a code PR, not a doc PR.
Full OS/container per-job sandboxing (§12)Explicitly out of scope per the issue.
Confirm scripts/gh-app-token behaviour on vps-i1 (§5)Needs a sys-admin session with live server access.