p24-infra Docs

plan-6618-bms4-autonomy-readiness

11 min read

Plan #6618 — bms-4 worker-fabric autonomy readiness

Status: design / awaiting /review-plan (min 3 cycles) before any Class-B/C implementation. Issue: radieu/p24-infra#6618 · Role authoring this plan: dev-coder. Goal: bring the bms-4 worker fabric to a state where it needs no interactive dev machine for the operations that failed on it recently (bms-1 OS upgrade 6602, IAM_ENC_KEY rotation #6561, future CF Worker deploys). The target boundary is “one-time paste of one token per external service” — not “zero dev-access”, which is impossible (see Class A).

1. Problem framing — three classes of barrier (verified live 2026-08-23)

The key enabling fact, confirmed against .sops.yaml: the bms-4 worker age key age1qrccz84pj8huj8hmx6452te2zugane0slgxs09h9s05m4zpgpetq8p9up0 is already a recipient of the default 6-recipient .sops.yaml rule, so the bms-4 infra-task worker can already sops -d every default-rule file directly. It is also the second recipient (with the developer key) on the two existing narrow worker files pinbox24-gitlab.env.sops and wasabi-admin-worker.env.sops. This means new scoped SOPS files can be encrypted to the worker from public keys alone — no developer age private key is required to create them.

ClassDefinitionAutonomous?Examples (this issue)
AValue origination behind a vendor panel gated by human 2FA. No API to mint.❌ never — one-time human pasteValid SoYouStart/OVH-budget dedicated-server API consumer-key for bms-1 (SYS_INFRA_* currently returns INVALID_KEY); a CF Workers Scripts:Edit token on the whatsup CF account; Wasabi root key rotation (no self-rotation API).
BDistribution of an originated value into scoped SOPS + secrets-sync wiring + re-key.✅ after origination — secret-manager on bms-4 already does this (proof: rotations 6593).SOPS scaffold precreation (§3), secrets-sync jobs.
CTooling + sanction gates entirely inside the repo.✅ nowdestructive-gate override token (§4), wrangler availability (§5), wa-app-api deploy pipeline (§5).

The residual human step this plan is aiming for is exactly: sops edit <scoped-file> → paste one freshly-minted token → git commit. Zero recipient-surgery, zero re-key, because the scoped files are pre-created (Class B, §3) with the worker already a recipient.

2. Scope, ownership, and gating (what ships where)

#ItemClassOwner roleGateShips in
§3SOPS scaffold precreation (ovh-sys, cloudflare-workers-bms4) + .sops.yaml rules + secrets-sync jobsBsecret-manager (SOPS write)after this plan approvedfollow-up secret-manager issue
§4Destructive-gate authorization-token convention in infra-task-request-worker.md Step 4C (security-sensitive)dev-coderhard-gated on green /review-plan (risk+security+compliance)follow-up dev-issue
§5wrangler globally on bms-4Csys-admin/infra-task (server op / Ansible)after this plan approvedfollow-up infra-task issue
§5wa-app-api (repo whatsup-android-chat-puller) deploy pipelineCdev-coder in that repoafter this plan approvedfollow-up cross-repo issue
§6Class-A human-origination checklistA (doc)dev-codernone — docthis PR
§7bms1-ubuntu-2204-upgrade.md runbook safety fix (wrong server-ID)— (doc)dev-codernone — docthis PR

This PR (dev-coder, plan issue) ships only §6 + §7 + this document. Everything in Classes B/C is out of dev-coder scope or hard-gated, so it is deferred to the follow-up issues listed above and tracked from the issue. Rationale: dev-coder capabilities.cannot forbids writing/decrypting any secrets/*.env.sops (§3 → secret-manager) and SSH/Ansible against a server (§5 wrangler → sys-admin); §4 is dev-coder-editable but the issue mandates it not be implemented without a green /review-plan.

3. Class B — SOPS scaffold precreation (follow-up: secret-manager)

Objective: collapse the human step for each Class-A credential to sops edit → paste → commit.

3.1 New scoped files (placeholders only, 2 recipients)

Model exactly on pinbox24-gitlab.env.sops / wasabi-admin-worker.env.sops — 2 recipients: developer key age1utmdm6cj2ray70g5nnupza8uvyuren26xk6a6nd4a58arax5xa7qaax7al + bms-4 worker key age1qrccz84pj8huj8hmx6452te2zugane0slgxs09h9s05m4zpgpetq8p9up0.

New filePlaceholder keysConsumerNotes
secrets/ovh-sys.env.sopsSYS_INFRA_APP_KEY, SYS_INFRA_SECRET, SYS_INFRA_CONSUMER_KEYbms-1 dedicated-server API ops (rescue/reboot/reinstall)See decision 3.3 — new file vs re-key of ovh-api.env.sops.
secrets/cloudflare-workers-bms4.env.sopsCF_WORKERS_EDIT_TOKENwrangler deploy of wa-app-api (p24-wa-app-api) on the whatsup CF accountscope = Workers Scripts:Edit for wa-app-api only.

Placeholder discipline (issue gate): the files must contain placeholder strings only, never real values (e.g. SYS_INFRA_APP_KEY=PLACEHOLDER_PENDING_HUMAN_PASTE). Canary-decrypt before commit. Per-key grep|cut, never source the whole .env (footgun class of 6600).

3.2 .sops.yaml rules

Add two path_regex rules above the default rule, byte-identical in shape to the existing pinbox24-gitlab / wasabi-admin-worker rules (developer + bms-4 worker key). These rules must be added before the files are encrypted (SOPS reads the rule to pick recipients at encrypt time).

3.3 Decision — new ovh-sys.env.sops vs re-key ovh-api.env.sops

Recommendation: create a new secrets/ovh-sys.env.sops; do not re-key ovh-api.env.sops.

  • The bms-4 worker can already decrypt ovh-api.env.sops (default-rule file) — so this is not about decryptability. It is about (a) a clean one-credential paste surface for the human, and (b) blast-radius isolation of the SoYouStart/budget-brand API family from the OVH-account API family that share ovh-api.env.sops today.
  • ovh-api.env.sops is excluded from secrets-sync.yml (!secrets/ovh-api.env.sops, line 30) and consumed by direct worker sops -d. A new isolated file keeps that “direct-decrypt, no CI push” property while giving a dedicated paste target.
  • Migration of the existing OVH_INFRA_* keys out of ovh-api.env.sops is out of scope — this plan only adds the SYS_INFRA_* scoped home. (Open question Q1 for review: should SYS_INFRA_* be removed from ovh-api.env.sops to avoid a two-copy drift, mirroring the GITLAB_ADMIN_PAT atomic-rotation policy? Default answer: yes, remove from ovh-api.env.sops in the same secret-manager PR so there is exactly one home.)

3.4 secrets-sync.yml wiring (deploy-guarded, fail-closed)

For any new file that must reach a server, add a dedicated sync-* job modeled on sync-traccar (.github/workflows/secrets-sync.yml), whose guard step fails closed while the value is still a placeholder:

# Guard — refuse deploy while the credential is still a placeholder (mirror of sync-traccar #5510)
VAL=$(grep -m1 '^SYS_INFRA_CONSUMER_KEY=' "$RUNNER_TEMP/ovh-sys.env" | cut -d= -f2-)
case "$VAL" in ""|PLACEHOLDER_*) echo "placeholder — refusing to deploy"; exit 1;; esac
  • ovh-sys.env.sops: no server push needed — the worker decrypts it directly on bms-4. So it is excluded from the push trigger (add !secrets/ovh-sys.env.sops to on.push.paths, same as ovh-api.env.sops). No sync-* job required.
  • cloudflare-workers-bms4.env.sops: same — the worker reads CF_WORKERS_EDIT_TOKEN directly at deploy time. Exclude from push trigger; no sync-* job. (Open question Q2: confirm the worker reads it via direct sops -d, not a CI-deployed plaintext, before deciding no job is needed.)
  • detect-changes path lists must be updated only if a sync-* job is actually added.

Note on workflow-file pushes: editing .github/workflows/secrets-sync.yml requires the GH_TOKEN PAT push path (docs/playbooks/worker-push-workflow-files.md) — the worker’s default gh OAuth login lacks workflow scope. The secret-manager follow-up must budget for this.

3.5 Compliance obligation (CLAUDE.md §Compliance)

§3 introduces two new scoped credential families (ovh-sys, cloudflare-workers-bms4) and, conditionally, new secrets-sync.yml jobs. Per CLAUDE.md §Compliance (“New … credential/workflow → update dev_r_services + ops doc”), the secret-manager follow-up must, as part of the same PR:

  • add the two files to the CLAUDE.md §SOPS file map table (canonical secret registry);
  • record any new secrets-sync.yml job / new credential family in dev_r_services where applicable, and reference the docs/policies/credential-isolation-policy.md two-copy/atomic-rotation policy if Q1 leaves SYS_INFRA_* duplicated.

This line exists so the delegated follow-up does not drop the compliance step.

4. Class C — destructive-gate authorization-token (follow-up dev-issue, review-gated)

Current behaviour (infra/agent-prompts/infra-task-request-worker.md Step 4): any of delete|drop|truncate|rm -rf|remove credential|revoke|wipe|purge|destroy in the issue title/body → hard block → human-action label → STOP. There is no override path, so a legitimately pre-authorized destructive op (e.g. do-release-upgrade on bms-1, dd a rescue image) can never run autonomously even when a human has explicitly sanctioned it.

Proposed convention (design only — do not implement without a green /review-plan):

  1. Override token line in the issue body: # DESTRUCTIVE-AUTHORIZED: <github-user> <issue#> <ISO-8601-timestamp>
  2. Mandatory rollback-proof requirement: the same issue must contain a Rollback: / rescue-path / backup reference (e.g. “OVH rescue image captured”, “mongodump at s3://…”). Gate refuses if absent.
  3. Step 4 lets the destructive payload through only when both the token line and the rollback-proof are present, and writes an audit-trail row (dev_r priorities/session log) recording who authorized what.
  4. Anti-self-authorization (review Q3): require issue-author == <github-user> on the token line and that the author is on an allow-list of authorized humans — so a worker cannot write its own override. The allow-list source is an open question (hard-coded vs a dev_r_* table vs GH team).

Why review-gated: this deliberately relaxes the single gate that protects production from autonomous destructive actions. It must pass /review-plan on the risk, security, and compliance perspectives before any code lands. Do not blind-ship. The infra-task-request-worker.md file is not protected: true, so dev-coder may edit it — but the review gate is a hard precondition per the issue.

5. Class C — tooling gaps

5.1 wrangler on bms-4 (follow-up: sys-admin/infra-task via Ansible)

Today CF Worker deploys in this repo run npx wrangler deploy (see deploy-meta-dispatcher.yml, deploy-p24-auth-worker.yml). For an infra-task worker deploying directly on bms-4, npx wrangler also works but re-downloads per invocation. Options:

  • (recommended) add npm install -g wrangler to the claude-runner Ansible role / bootstrap-new-vps.sh Phase 2 (next to the existing global @anthropic-ai/claude-code install), so every runner host has it. Server op → sys-admin/infra-task.
  • (fallback) keep npx wrangler and accept per-invocation download. No change needed.

This plan recommends the Ansible install for determinism, delegated to sys-admin.

5.2 wa-app-api deploy pipeline (follow-up: cross-repo, dev-coder in whatsup-android-chat-puller)

p24-wa-app-api is a Cloudflare Worker living in the separate repo radieu/whatsup-android-chat-puller — its deploy workflow cannot be created in p24-infra. Spec (model on .github/workflows/deploy-p24-auth-worker.yml):

  • Trigger: push to the worker’s source path on main (+ workflow_dispatch).
  • Auth: CLOUDFLARE_API_TOKEN = the scoped CF_WORKERS_EDIT_TOKEN from §3, delivered as a GH Actions secret in that repo (or decrypted from the scoped SOPS file by the bms-4 worker for a worker-driven deploy).
  • Step: npx wrangler deploy in the worker directory.
  • Standard error notification (Discord + GH issue) on failure.

Deliverable: a follow-up issue in radieu/whatsup-android-chat-puller referencing this spec.

6. Class A — human-origination checklist (ships in this PR)

One-time, human-with-panel-access steps. After each paste, the named issue becomes fully autonomous.

CredentialWhere to originateExact actionPaste targetUnblocks
SoYouStart/budget dedicated-server API consumer-key (SYS_INFRA_*)the budget-brand control panel that actually owns bms-1 (SoYouStart or Kimsufi — verify which; not the standard OVH Manager) — createToken endpoint, e.g. https://eu.api.soyoustart.com/createToken/mint token with scope GET/POST/PUT /dedicated/server/*; capture APP_KEY/SECRET/CONSUMER_KEYsops edit secrets/ovh-sys.env.sops (after §3)bms-1 rescue/reboot/reinstall automation (#1756/#6149/#6602)
CF Workers Scripts:Edit token (whatsup account)Cloudflare dashboard of the whatsup account → My Profile → API Tokenscreate token, template “Edit Cloudflare Workers” scoped to wa-app-apisops edit secrets/cloudflare-workers-bms4.env.sops (after §3)autonomous wa-app-api deploys
Wasabi root key rotationWasabi console (no self-rotation API)rotate per docs/playbooks/wasabi-key-rotation.md §Human-actionexisting wasabi-admin-worker.env.sops (secret-manager)Wasabi root rotation

7. Runbook safety fix (ships in this PR)

docs/playbooks/bms1-ubuntu-2204-upgrade.md currently instructs the day-of operator to open “OVH Manager / IPMI-KVM access (server ID 1823494)”. #6618 flagged (live 2026-08-23) that 1823494 is a different server (bms-2, a MongoDB rs0 PRIMARY) and that bms-1’s console lives in a budget-brand panel, not the standard OVH Manager. Opening rescue/KVM on the wrong server during a production maintenance window is a real outage risk.

Conservative fix applied here (I cannot verify bms-1’s exact server-ID or exact panel brand without panel/API access — that is a sys-admin/human step, so I do not invent a replacement number):

  • Remove the hard-coded server ID 1823494 and replace it with a ⚠️ verify-before-KVM step that explicitly warns 1823494 was found to belong to a different server (bms-2) and must not be used.
  • Soften “OVH Manager / IPMI-KVM” to “the budget-brand control panel that owns bms-1 (verify: Kimsufi vs SoYouStart — not the standard OVH Manager)”.
  • Cross-reference #6618 so the authoritative server-ID/panel confirmation is tracked.

Provenance caveat. The 1823494 == bms-2 and “SoYouStart not OVH Manager” claims come from the #6618 live finding (2026-08-23) and were not independently re-verified by this dev-coder worker (no panel/API access in role). The runbook edit is therefore written as a conservative hedge (“do not use 1823494; confirm in the panel first”) rather than asserting a specific replacement ID/brand — the authoritative confirmation is a sys-admin/human step tracked on #6618.

8. Regression risks

  • §7 runbook edit — docs-only; risk is under-correcting (leaving ambiguity). Mitigated by the explicit “verify in panel, do not use 1823494” warning, which is strictly safer than the current wrong-ID instruction.
  • §4 (deferred) — highest-risk item; relaxes prod-protection. Mitigated by the hard /review-plan gate and the anti-self-authorization requirement.
  • §3 (deferred) — SOPS write; risk of committing a real value or a corrupt SOPS file. Mitigated by placeholder-only + canary-decrypt discipline, owned by secret-manager.

9. Manual test checklist

  • docs/plans/plan-6618-bms4-autonomy-readiness.md renders (tables, headings).
  • docs/playbooks/bms1-ubuntu-2204-upgrade.md no longer contains the string 1823494 as a bms-1 instruction; contains the verify-before-KVM warning.
  • /review-plan run ≥3 cycles; risk/security/compliance perspectives addressed for §4.
  • Follow-up issues created: secret-manager (§3), dev-issue (§4, review-gated), infra-task (§5.1), cross-repo (§5.2).

10. Open questions for /review-plan

  • Q1 — remove SYS_INFRA_* from ovh-api.env.sops when adding ovh-sys.env.sops (single-home) vs leave both (two-copy drift)? Default: single-home.
  • Q2 — does the bms-4 worker read the new scoped CF/OVH files via direct sops -d (no sync-* job) or via CI-deployed plaintext (job needed)?
  • Q3 — anti-self-authorization allow-list source for §4 (hard-coded / dev_r_* table / GH team)?

files_to_change (this PR): docs/plans/plan-6618-bms4-autonomy-readiness.md, docs/playbooks/bms1-ubuntu-2204-upgrade.md

Graph View