p24-infra Docs

plan-4065-github-pat-autonomous-rotation

21 min read

Plan: enable autonomous GitHub PAT rotation (TOTP + Playwright) on the secret-manager worker

Issue: #4065 Type: Code-change-design (plan only) Status: Draft — iteration 1 Last updated: 2026-07-13 Author: Claude Opus 4.7 (worker session on vps-h1) Related: #4053 (blocked et-op GITHUB_TOKEN rotation), #2824 (et-op key exposure — GITHUB_TOKEN still unrotated), #4048 (.claude-env scrub), #3286 (recurring GH_TOKEN leak via git remote -v), #3304 (git credential helper on bms-4 — closes the leak vector but does not enable rotation)

0. Summary — recommendation

Ship a two-track enablement, in order:

  1. Wave A — GitHub App expansion (primary): move as much automation as possible off PATs onto the existing GitHub App (GITHUB_APP_ID=2109526, installation 142989418). The App produces 1h-TTL installation tokens with no browser and no 2FA — this is Tier 1 autonomy today. Coverage gap analysis (§4) shows the App can absorb every current PAT consumer except GH_PAT_ADMIN (admin-scope operations outside App permission surface). Delivered by a p24-infra PR that adds a bin/gh-app-token.sh helper, refactors the four bms-4 worker code paths that still call PATs directly, and revises the access matrix.
  2. Wave B — Playwright+TOTP capability (residual fallback): only after Wave A ships, provision Playwright + Chromium on the bms-4 secret-manager worker, add GITHUB_TOTP_SECRET to secrets/n8n-bms4.env.sops, and extend worker-secret-manager.md with the Playwright login flow. This unblocks the small residual set (Wave A leftovers + any provider that never adopts the App) and unblocks #4053 immediately once the TOTP seed is added.

Rationale for the ordering: Wave A removes ~90% of the PAT-rotation surface without introducing any browser/2FA dependency on the worker, and the App private key already sits in n8n-bms4.env.sops (verified — see §2.4). Wave B is unavoidable for GH_PAT_ADMIN and for the fine-grained PAT under et-op (#4053), but is a larger blast radius (browser + TOTP seed → worker environment), so it should not gate the App migration.

GitHub App alternative decision: adopt as primary path (Wave A). Do not treat it as a replacement for Wave B — the App cannot regenerate GH_PAT_ADMIN nor create PATs itself. Wave B stays scoped to the PAT-only residual set.

Not shipped in this PR: the actual Playwright script, the SOPS entry for GITHUB_TOTP_SECRET (secret-manager worker task, not this plan-PR), and the re-dispatch of #4053. Those follow after this plan is approved and the two implementation issues (§6.4) land.

1. Why this is blocked today — reproduction of the #4053 callback failure

The bms-4 secret-manager worker attempted to rotate et-op GITHUB_TOKEN (fine-grained PAT, Tier 2) on 2026-07-12 and reported two blockers:

BlockerEvidenceConsequence
B1 — GITHUB_TOTP_SECRET absentgrep -l '^GITHUB_TOTP_SECRET=' secrets/*.env.sops returns 0 files. Also missing from .env.local (development-local backstop).Playwright cannot complete GitHub 2FA challenge → login flow dies at OTP prompt. Access matrix line 141 says “prerequisite unmet → escalate to Tier 3 human action.”
B2 — Playwright/Chromium unavailable on the workeransible/roles/claude-runner/tasks/main.yml installs Node.js + Claude Code CLI only. No playwright install chromium, no browser binary, no per-role node_modules. scripts/rotate/ has .js scripts that call require('playwright') but no package.json at that path in the checkout.Even with the TOTP seed, the worker has no browser to drive. Fine-grained PAT regeneration is UI-only — no API.

Fine-grained PATs (github_pat_* prefix) have no rotation API — the only interface is github.com/settings/personal-access-tokens/<id> → Regenerate button, guarded by full password login + 2FA. This is by GitHub design and will not change.

Corollary: every future PAT rotation currently degrades to human-action, blocking the SLA watchdog and the exposure-response protocol. #4053 is the acute case; #2824’s remaining 11 exposed keys are the standing case.

2. Current state — facts on the ground

2.1 PAT inventory (across the fleet)

Verified via grep -nE '^GH_TOKEN|^GITHUB_PAT_|^GITHUB_TOKEN' secrets/*.env.sops on branch head:

Key nameSOPS fileDeployed toConsumer / purpose
GH_TOKENsecrets/monitoring.env.sops (line 8)vps-i1 /opt/p24-infra/monitoring/.envMonitoring exporters, cron scripts on vps-i1
GH_TOKENsecrets/n8n-bms4-gh.env.sops (line 1)bms-4 /opt/p24-infra/bms-4/n8n-bms4-gh.envgit credential helper on bms-4 (git-credential-gh-token.sh) — closes #3286 leak
GITHUB_PAT_ALL_WRITESsecrets/n8n-bms4-gh.env.sops (line 2)bms-4n8n workflows: issue/PR create, dispatch triggers, workflow triggers
GITHUB_PAT_P24_INFRAsecrets/n8n-bms4-gh.env.sops (line 3)bms-4n8n workflows scoped to radieu/p24-infra
GITHUB_TOKENsecrets/et-operational-platform.env.sops (line 23)Vercel (et-operational-platform project)et-op backend, job_type=secret-manager dispatch producer
GH_TOKEN (playbook stub)secrets/vps-h1.env.sopsvps-h1Older scripts (WAHA-era) — grep confirms not present today, playbook is stale
GH_PAT_ADMINsecrets/administration.env.sopsDeveloper workstation onlyOwner-scope admin operations. Worker cannot touch this file.

Playbook drift found: docs/playbooks/github-pat-rotation.md line 18 says the three bms-4 PATs live in secrets/n8n-bms4.env.sops, but the actual location is secrets/n8n-bms4-gh.env.sops. Wave A PR must correct this (see §6.2). Also the vps-h1 row in playbook table 4 is stale — no PAT ships there today.

2.2 GitHub App inventory (already in place)

Verified via grep -nE '^GITHUB_APP' secrets/n8n-bms4.env.sops:

KeyLinePurpose
GITHUB_APP_ID432109526 (n8n.io pull-request-flow App)
GITHUB_APP_PRIVATE_KEY_B6444Base64-encoded RSA PEM (used by n8n sub-workflow 6b7m52mpur3tXO1G and by docs/playbooks/github-app-token-n8n.md)
GITHUB_APP_INSTALLATION_ID45142989418 — radieu account, all repos

The App is installed on the radieu GitHub account. docs/playbooks/github-app-token-n8n.md §Comparison confirms: 1h installation tokens, no browser, no 2FA, per-request scoping. The plumbing already works for n8n; the gap is that the bms-4 worker code paths (secrets-sync helper, gh CLI shell-outs, scripts/rotate/*) still use PATs directly.

2.3 What the PATs are actually used for (consumer census)

The point of Wave A is to cut the PAT-consumer count. Enumerate every place PATs are read:

ConsumerReads which PATReason for PAT (vs App)App-compatible?
bms-4 git-credential-gh-token.sh (PR #3304)GH_TOKENgit clone / git push over HTTPS✅ App installation tokens work as HTTPS Basic-auth passwords with username x-access-token
bms-4 secrets-sync.yml “Ship to bms-4” stepGH_TOKEN (via GH Secret)Push branch update to /opt/p24-infra remote✅ swap to App token generated at CI runtime
n8n workflows on bms-4GITHUB_PAT_ALL_WRITES, GITHUB_PAT_P24_INFRAIssue create, PR comment, workflow dispatch✅ App can perform all of these (Contents: RW, Issues: RW, PRs: RW, Actions: RW — already in App manifest per §2.4 verification)
vps-i1 monitoring cronGH_TOKENOpen GH Issue on error (per §Error Notification Standard)✅ App can POST /repos/.../issues
et-op backendGITHUB_TOKEN (fine-grained PAT)Queue-dispatch producer + repo webhooks⚠️ Feasible but riskier (Vercel side needs App-JWT signer library at runtime); may be deferred to Wave B
Developer workstationGH_PAT_ADMINOwner-scope operations (Apps management, org billing)❌ Out of App permission surface — Wave B only
bms-4 scripts/rotate/*.js (Playwright rotations)GH_TOKENgh secret set after rotation✅ App token for the gh secret set call

Net: four out of seven consumer classes migrate cleanly to the App in Wave A. et-op backend and GH_PAT_ADMIN remain PAT-bound and drive the Wave B requirement.

2.4 Verify the App has the permissions we need — DO before Wave A PR

Wave A depends on the installed App already having the permission surface listed in §2.3. Verification is a one-shot API call (no browser):

# From any host with GITHUB_APP_PRIVATE_KEY_B64 accessible (bms-4 or vps-i1):
TOKEN=$(python3 -c "
import base64, jwt, time, os
pem = base64.b64decode(os.environ['GITHUB_APP_PRIVATE_KEY_B64']).decode()
now = int(time.time())
print(jwt.encode({'iat': now - 60, 'exp': now + 540, 'iss': os.environ['GITHUB_APP_ID']}, pem, 'RS256'))
")
curl -sf -H "Authorization: Bearer $TOKEN" \
  -H "Accept: application/vnd.github+json" \
  "https://api.github.com/app/installations/${GITHUB_APP_INSTALLATION_ID}" \
  | jq '.permissions,.repository_selection'

Expected permissions surface (App manifest must include): contents: write, issues: write, pull_requests: write, actions: write, metadata: read, secrets: write (for the gh secret set migration path). If any are missing, the App manifest gets updated at github.com/settings/apps/2109526/permissions — that is a Tier 3 human action (§Escalation) and this plan’s Wave A becomes gated on it. Verify before the Wave A PR opens so the reviewer can decide whether to split.

Do NOT print the JWT or installation token to stdout — pipe directly into curl, unset immediately.

2.5 Existing Playwright surface in the repo

scripts/rotate/*.js files exist for 8 credentials (discord-*.js, n8n-bms4-api-key.js, mailgun-api-key.js, etc.). All follow the pattern in docs/playbooks/playwright-rotation-template.md:

  • Executed from a Windows dev machine (existing .ps1 wrappers assume %LOCALAPPDATA%\ms-playwright).
  • Reuse a mcp-chrome-* Playwright profile — persistent Google session, no fresh login.
  • No 2FA handler — the ones that need MFA (n8n-bms4-api-key.js) exit code 2 for the caller to retry.

None of these run on the bms-4 worker today. The prompt in docs/playbooks/github-pat-rotation.md §Option A describes the intent of the Playwright-on-worker flow, but the infrastructure (browser install, headless mode, TOTP handler, ephemeral Chrome profile) is not provisioned. Wave B provides that.

2.6 Account ownership — which GitHub account owns each PAT

Issue body asks whether all PATs are on the same account (single TOTP unlocks everything). Sampled evidence:

  • GH_TOKEN on bms-4 → 2026-07-08 rotation-log entry confirms radieu account (gh api /user returned radieu).
  • GITHUB_PAT_ALL_WRITES / GITHUB_PAT_P24_INFRA → n8n-bms4-gh.env.sops proximity + rotation history in github-pat-rotation.md → radieu account.
  • GITHUB_TOKEN (et-op) → per #4053 body, fine-grained PAT on radieu account.
  • GH_PAT_ADMIN → per access-matrix line 172, on radieu (admin-scope PAT).

Conclusion: a single GITHUB_TOTP_SECRET seeded from radieu’s authenticator unlocks all PAT UI operations. No dual-account fan-out.

Confirm in the Wave B implementation issue by having the human read the QR seed once at github.com/settings/security → 2FA reconfiguration flow, then verify a manual TOTP code against the seed before it lands in SOPS.

3. Design

3.1 Wave A — GitHub App migration for bms-4 workers (this-PR-adjacent implementation issue)

Goal: eliminate PAT dependency for 4 of 7 consumer classes.

Deliverables (opened as separate implementation issue tbd — see §6.4):

  1. bin/gh-app-token.sh — new POSIX shell helper (bms-4 + vps-i1). Signs a 9-min JWT with GITHUB_APP_PRIVATE_KEY_B64, exchanges for an installation token via POST /app/installations/{id}/access_tokens, prints the token to stdout with --no-newline. Callers use $(bin/gh-app-token.sh) inline. Same UX contract as an env-var read but never touches disk. Also exposes --check (verify creds work) and --repos <csv> (scope the token to specific repos).
  2. Refactor bms-4 git-credential-gh-token.sh (added by PR #3304) to call bin/gh-app-token.sh instead of reading GH_TOKEN from /opt/p24-infra/bms-4/n8n-bms4-gh.env. Fall back to GH_TOKEN only if the App call fails (belt-and-braces during rollout — remove the fallback in a follow-up after 2 weeks of clean logs).
  3. Refactor secrets-sync.yml “Ship to bms-4” step to run bin/gh-app-token.sh --repos radieu/p24-infra --scope contents:write on the runner and use that token for the deploy push. Removes the x-access-token:${GH_PAT}@github.com embedded-URL anti-pattern from CI (currently reinstalled every sync per #3286 root cause).
  4. Refactor scripts/rotate/*.js post-rotation gh secret set calls to source the App token instead of the ambient GH_TOKEN. Wrap as a single helper — scripts/lib/gh_app_secret_set.sh — so future scripts inherit the pattern.
  5. n8n workflow migration: audit the 8 n8n workflows currently reading GITHUB_PAT_ALL_WRITES / GITHUB_PAT_P24_INFRA (list in §6.3 acceptance criteria), switch each to the GitHub App Token Generator sub-workflow (ID 6b7m52mpur3tXO1G — already exists per docs/playbooks/github-app-token-n8n.md).
  6. Access matrix update — move GH_TOKEN, GITHUB_PAT_ALL_WRITES, GITHUB_PAT_P24_INFRA from Tier 2 to a new “Tier 1 via GitHub App” row that points at bin/gh-app-token.sh. Deprecate the PATs (mark for removal from SOPS after Wave A rollout + 30 days of no consumer usage).

Non-goal (deferred to Wave B): removing GITHUB_TOKEN (et-op) from Vercel or migrating et-op backend to App tokens. The Vercel side needs a serverless-friendly JWT signer, which is a separate design.

Rollback: if the App token call fails at any point, callers already fall back to the PAT read (delivered as belt-and-braces). Removing the App migration is git revert of the two commits.

3.2 Wave B — Playwright + TOTP capability for the secret-manager worker

Goal: enable autonomous rotation for the residual PAT-only set: et-op GITHUB_TOKEN (#4053), any future new PAT, GH_PAT_ADMIN (with an explicit --allow-admin guard).

Deliverables (opened as separate implementation issue tbd — see §6.4):

3.2.1 Provisioning — ansible/roles/claude-runner

Add tasks to ansible/roles/claude-runner/tasks/main.yml:

# Package list targets Ubuntu 22.04 (bms-4's actual OS). libasound2 (not the
# 24.04+ libasound2t64 t64-transition name) is correct for 22.04 — the
# original list mixed 22.04 and 24.04 package names and was uninstallable on
# either (#4956).
- name: Install Playwright system dependencies (bms-4 only — playwright_enabled hostvar)
  apt:
    name:
      - libnss3
      - libatk-bridge2.0-0
      - libdrm2
      - libxkbcommon0
      - libxcomposite1
      - libxdamage1
      - libxrandr2
      - libgbm1
      - libpango-1.0-0
      - libasound2
      - fonts-liberation
    state: present
  when: playwright_enabled | default(false)
 
- name: Install Playwright + Chromium via npm (global, headless)
  community.general.npm:
    name: playwright
    version: "1.48.0"     # pin — automation flakes on unexpected DOM shifts
    global: true
  when: playwright_enabled | default(false)
 
# --with-deps dropped (#4956): it shells out to `sudo apt-get install`
# internally, but claude-runner has no general sudoers grant on bms-4 (only a
# single scoped sops-install rule) — non-interactive sudo is denied. Also
# redundant: the apt task above already installs those deps as root.
- name: Install Chromium browser via Playwright
  command: playwright install chromium
  become: true
  become_user: claude-runner
  args:
    creates: /home/claude-runner/.cache/ms-playwright/chromium-*
  when: playwright_enabled | default(false)
 
- name: Ensure TOTP CLI (oathtool) is present for MFA challenges
  apt:
    name: oathtool
    state: present
  when: playwright_enabled | default(false)
 
# github-pat.js requires('playwright') as a LOCAL dependency (see
# scripts/rotate/package.json) — the global npm install above cannot satisfy
# Node's local require() resolution, so this task is required or the script
# throws MODULE_NOT_FOUND at runtime (#4956).
- name: Install scripts/rotate local npm dependencies (playwright local require)
  community.general.npm:
    path: /opt/p24-infra/scripts/rotate
    ci: true
  when: playwright_enabled | default(false)

Gate on playwright_enabled=true in inventory/host_vars/bms-4.yml. Do not enable on vps-i1 or vps-h1 — the Playwright capability is currently only needed on the secret-manager worker (bms-4).

Disk-cost impact: Chromium ~180 MB + system deps ~200 MB, negligible on the bms-4 40 GB disk (current use ~28%). RAM cost is per-run only (Playwright forks a browser process on demand); worker memory budget unchanged.

3.2.2 GITHUB_TOTP_SECRET — where and how

  • SOPS file: secrets/n8n-bms4.env.sops. Rationale: bms-4 is where the secret-manager worker runs; n8n-bms4.env.sops already deploys to /opt/p24-infra/bms-4/.env; the worker can source it via Step 0a of worker-secret-manager.md. Do not put it in secrets/administration.env.sops (worker is barred from that file per §Role Enforcement matrix).
  • Also mirror to vps-h1? No — vps-h1 does not run the secret-manager worker (see §Role Enforcement; secret-manager is bms-4-only). Single-file placement is correct.
  • Format: base32-encoded seed (raw GitHub 2FA seed as shown in the 2FA reconfiguration flow). Consumers use oathtool --totp -b "$GITHUB_TOTP_SECRET" to generate the 6-digit code at login time.
  • Ownership caveat: this seed grants anyone with SOPS+age keys the ability to bypass 2FA on radieu’s GitHub account for the PAT UI. This is the same trust boundary as the App private key already sitting in the same file. Access matrix line for GITHUB_TOTP_SECRET = Tier 2 (Playwright unlock) with note “seed grants 2FA-bypass for radieu account — treat like a root credential.”
  • Addition procedure: human obtains the seed at github.com/settings/security → 2FA → reconfigure authenticator → reveal seed. Then queue a secret-manager job with Operation: add (Wave B implementation issue posts the queue payload). Worker performs the SOPS write per worker-secret-manager.md §General pattern. The plan-PR does not carry the value — only the plan.

3.2.3 Playwright login flow — scripts/rotate/github-pat.js

New file, following the pattern in docs/playbooks/playwright-rotation-template.md §Automated Playwright script standard.

StepAction
0Read GITHUB_USERNAME (env; default radieu@gmail.com), GITHUB_PASSWORD (env, read from SOPS silently), GITHUB_TOTP_SECRET (env), PAT_NAME (env — which PAT to regenerate), TOKEN_OUT_FILE (env).
1Launch headless Chromium (--no-sandbox — bms-4 runs as claude-runner non-root, sandbox denied). Fresh context, no persistent profile (unlike the Windows scripts — no session to reuse on a headless worker).
2page.goto('https://github.com/login'), fill username + password.
3On 2FA page: generate OTP via require('otplib').authenticator.generate(GITHUB_TOTP_SECRET) (otplib is a peer dep, add via npm), fill in the 6-digit box.
4page.goto('https://github.com/settings/personal-access-tokens') → find PAT by name → Regenerate → set expiry 90 days → confirm.
5Extract the new token from the readonly input on the confirmation page (readonly <input> selector, fallback to code element per template contract).
6Write to TOKEN_OUT_FILE, clear in-memory variable, exit 0.
Exit codes0 success · 1 fatal · 2 MFA-required-but-no-secret (safe retry).

The script itself is scripts/rotate/github-pat.js (~150 LOC — reference the existing discord-bot-token.js closely). Add screenshot capture behind DEBUG_SCREENSHOTS=1 per template.

3.2.4 Wrapper — extend worker-secret-manager.md

Add a new “Tier 2 — Playwright rotation pattern (GitHub PAT)” section between the existing Tier 1 and Rotation-log sections. Contains:

  • Reading the target PAT’s SOPS location from the access matrix.
  • Sourcing GH_TOKEN / GITHUB_PAT_* / GITHUB_TOKEN old value silently for the “verify old key rejected” post-check.
  • Invoking scripts/rotate/github-pat.js with the right env.
  • Same SOPS write pattern as Tier 1 (§Linux worker -tmp.env.sops pattern), same canary, same distribution chain (GH Secret + live env + container recreate).
  • Revocation of the old PAT: skipped (fine-grained PAT UI has no revoke endpoint that Claude can call — the Regenerate action invalidates the old value automatically, verified in docs/2026-07/secrets-rotation-log-2026-07-08.md note “rotation ≠ revocation” is about the scheduled rotation of the pre-existing token, not about Regenerate itself).

3.2.5 Isolation and safety

  • Playwright runs in a fresh Chromium context per invocation (no userDataDir). Credentials are only in env vars, never on disk (no .env.local write, no cookie persistence).
  • All env vars unset immediately after node scripts/rotate/github-pat.js returns (bash unset, not export -n).
  • TOKEN_OUT_FILE is created with mktemp -t github-pat-out.XXXXXX, chmod 600 before write, shred -u after read.
  • Discord + GH issue error notification on any non-zero exit (per §Error Notification Standard).

Not doing: a separate container for the browser (isolation via subprocess is sufficient for this threat model; a container adds provisioning surface and doesn’t materially reduce blast radius since the worker itself holds the credentials).

3.3 E2E validation on a low-risk PAT

Before running Wave B against et-op GITHUB_TOKEN, validate on a fresh disposable PAT:

  1. Human creates a throwaway fine-grained PAT TEST_PAT_ROTATION_4065 on radieu account, scope: read-only Metadata on a single test repo. Records value in a temporary local env only.
  2. Add TEST_PAT_ROTATION_4065 to secrets/n8n-bms4.env.sops via a scratch secret-manager task.
  3. Dispatch a secret-manager rotation job for TEST_PAT_ROTATION_4065.
  4. Worker runs the new Playwright flow, produces a new value, updates SOPS, verifies the new value via curl -H "Authorization: Bearer $NEW" api.github.com/user.
  5. Log the run in docs/secrets-rotation-log.md marked test-pat validation-run.
  6. Remove TEST_PAT_ROTATION_4065 from SOPS + delete the PAT at GitHub.

Acceptance criteria for the whole enablement: this E2E run passes end-to-end with zero human interaction after step 1. If it needs manual intervention, roll back the Wave B PR and open a follow-up.

3.4 Re-dispatch #4053

Post-validation, the Wave B implementation issue’s final step:

curl -sf -X POST "$QUEUE_API_URL/queue-issue" \
  -H "Authorization: Bearer $QUEUE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"issue_number": 4053, "repo": "radieu/p24-infra", "job_type": "secret-manager"}'

The bms-4 secret-manager worker picks it up, sees GITHUB_TOTP_SECRET present + Playwright available, proceeds through Tier 2, rotates the et-op GITHUB_TOKEN, closes the loop on #2824’s remaining GITHUB_TOKEN PAT unrotated line. #4053’s deferral note is cleared in the same worker run.

4. Alternative evaluated: GitHub App exclusively (skip Wave B)

Considered. Rejected as sole solution — kept as Wave A primary.

Why not App-only:

  • GH_PAT_ADMIN operations (App management itself, org billing, some organization admin API paths) are outside the App’s permission surface by design. Never fixable via App scoping.
  • Vercel deployments (et-operational-platform) currently read GITHUB_TOKEN at runtime for webhook validation and downstream automation. Switching to App tokens requires a runtime JWT signer + refresh loop in serverless, which is a larger project. Wave B unblocks the rotation today without waiting for the Vercel refactor.
  • Rotating the App private key itself (GITHUB_APP_PRIVATE_KEY_B64) is a Tier 3 human action per access-matrix line 166 — the private-key generation UI is browser+2FA-gated. So even the “App everywhere” world does not eliminate the browser-login capability requirement — it just shifts it from PATs to the App private key. Wave B is therefore not “extra work” — it is work that would be needed regardless the day the App key rotates (expected annually).

Why Wave A is still worth doing: cuts the PAT-rotation cadence from ~4 rotations/year (per key × 3 keys) to ~1 rotation/year (App key only). Every consumer moved off PATs is one fewer thing to remember when the next PAT expires.

5. Files to change

5.1 This plan PR (only)

  • NEW docs/plans/plan-4065-github-pat-autonomous-rotation.md — this file.
  • UPDATE docs/priorities.md — NO CHANGE (per CLAUDE.md: docs/priorities.md is human-supervised only; do not edit from a worker).

Nothing else. This PR does not carry code, playbook edits, secrets, or ansible tasks. Those follow in the Wave A / Wave B implementation PRs (§5.2, §5.3).

5.2 Wave A implementation PR — files preview

  • NEW bin/gh-app-token.sh — JWT signer + installation token exchange helper (POSIX shell + python for the JWT step).
  • UPDATE bms-4/git-credential-gh-token.sh — try App token first, fall back to GH_TOKEN.
  • UPDATE .github/workflows/secrets-sync.yml — replace embedded-URL push with App-token push in the “Ship to bms-4” step.
  • NEW scripts/lib/gh_app_secret_set.sh — App-authed gh secret set wrapper.
  • UPDATE scripts/rotate/*.js (~8 files) — switch post-rotation secret-set calls to the new helper.
  • UPDATE n8n workflows (8 items, exported via n8n API + committed to infra-src/n8n-workflows/) — replace PAT credential nodes with GitHub App Token Generator sub-workflow calls.
  • UPDATE docs/playbooks/secret-rotation-access-matrix.md — move GH_TOKEN / GITHUB_PAT_* from Tier 2 to Tier 1-via-App; keep the Tier 2 row for GH_PAT_ADMIN only.
  • UPDATE docs/playbooks/github-pat-rotation.md — correct the SOPS file (n8n-bms4-gh.env.sops, not n8n-bms4.env.sops) and add an “App-first” section that supersedes Option A for non-GH_PAT_ADMIN PATs.

5.3 Wave B implementation PR — files preview

  • UPDATE ansible/roles/claude-runner/tasks/main.yml — Playwright + Chromium install tasks (per §3.2.1).
  • UPDATE ansible/inventory/host_vars/bms-4.yml — set playwright_enabled: true.
  • NEW scripts/rotate/github-pat.js — Playwright automation (per §3.2.3).
  • NEW scripts/rotate/package.json + scripts/rotate/package-lock.json — declare playwright@1.48.0 and otplib (they’re currently referenced but not tracked).
  • UPDATE infra/agent-prompts/worker-secret-manager.md — new Tier 2 section (per §3.2.4).
  • UPDATE docs/playbooks/github-pat-rotation.md — replace Option A prose with the new worker-side flow (points at scripts/rotate/github-pat.js).
  • UPDATE docs/playbooks/secret-rotation-access-matrix.md — add GITHUB_TOTP_SECRET row (Tier 2 with the safety note).
  • NEW SOPS ENTRY GITHUB_TOTP_SECRET in secrets/n8n-bms4.env.sops — added by the secret-manager worker in the same PR (not by the plan author). Value provided by radieu at issue-creation time.

5.4 New GitHub Issues opened by this plan

The plan-PR does not implement — it queues follow-up issues:

  • Issue A (Wave A): “[plan-child] Migrate bms-4 workers off GitHub PATs to installation tokens” — implements §3.1 / §5.2. Labels: enhancement, refs #4065.
  • Issue B (Wave B): “[plan-child] Provision Playwright + GITHUB_TOTP_SECRET on bms-4 secret-manager worker” — implements §3.2 / §5.3. Labels: enhancement, secret-manager, refs #4065, #4053.
  • Issue C (E2E validation): “[plan-child] Validate autonomous GitHub PAT rotation on TEST_PAT_ROTATION_4065” — implements §3.3. Blocks on Issue B. Refs #4065.
  • Issue D (Re-dispatch #4053): worker action inside Issue C (no separate issue); tracked as a comment on #4053.

Opened by the human after this plan-PR merges (or by the worker at plan-approval — decide during review).

6. Regression risks

RiskWhereMitigation
App token has insufficient permissions for a consumer that PATs handled fineWave A — n8n workflow migrations§2.4 verification step before PR; keep GH_TOKEN fallback in git-credential-gh-token.sh during rollout.
GITHUB_TOTP_SECRET seed in SOPS grants 2FA bypass on radieu GitHub account to anyone with age keysWave BSame threat model as the App private key already in n8n-bms4.env.sops. Access matrix labels it Tier-2-with-safety-note. Not a new trust surface.
Playwright DOM selectors break on next GitHub UI redesignWave B scripts/rotate/github-pat.jsPin playwright@1.48.0. Add DEBUG_SCREENSHOTS mode. On selector failure exit code 1 → human-action fallback (same behavior as today’s blocked state).
Headless Chromium runs as non-root claude-runner without sandbox → potential local sandbox-escapeWave B — --no-sandbox in Chromium launchThreat model: browser drives only github.com (fixed navigation, no user-controlled URLs). Sandbox-escape risk is bounded by GitHub content trust, which we already trust for all our git operations. Acceptable.
Rotation of TOTP seed itself (if compromised)Wave B ongoingSame as any other Tier 2 credential — re-run the human step of 2FA reconfiguration, re-add to SOPS. Playbook add-on to §3.2.4 covers this.
GitHub PAT UI DOM regressions between the two Waves — during the gap window, PATs are still Tier 2, still blockedBetween Wave A merge and Wave B mergeWave A does not depend on Wave B — no coupling. If Wave B lags, PAT rotations remain Tier 2 (as today) but Wave A consumers no longer need those PATs to work day-to-day.
SOPS canary breaks after GITHUB_TOTP_SECRET addWave B implementationMandatory canary in worker-secret-manager pattern; canary before rename per §Linux worker SOPS pattern.
PR to main (not dev) — this repo requires itThis plan PRConfirmed per CLAUDE.md §Branching — all PRs target main directly.

7. Manual test checklist (for plan review — not for implementation)

  • Reviewer confirms Wave A is worth doing before Wave B (do not skip A to jump to B).
  • Reviewer confirms the App permission surface via §2.4 API call (or requires that check in the Wave A PR before merging).
  • Reviewer confirms secrets/n8n-bms4.env.sops is the right home for GITHUB_TOTP_SECRET (bms-4 residency, not administration).
  • Reviewer confirms Playwright pin (1.48.0) vs “latest” — pin recommended.
  • Reviewer approves opening the four follow-up issues (§5.4) as a batch after merge.
  • Reviewer approves the deferred et-op-Vercel App-migration scope (Wave A explicitly does not attempt it).

8. Dependencies

  • No external services required for this plan-PR. Implementation PRs (Wave A / Wave B) will need:
    • Wave A: python3-jwt on bms-4 + vps-i1 (already present per ansible/roles/claude-runner).
    • Wave B: apt packages listed in §3.2.1 + npm registry access from bms-4 (already available — claude-runner installs Claude Code CLI at boot).
  • Blockers upstream: none. GITHUB_APP_ID + GITHUB_APP_PRIVATE_KEY_B64 are already in place (§2.2). Wave B needs one Tier 3 human step (radieu provides TOTP seed) before the implementation issue can complete.

9. Non-scope

  • Vercel et-op runtime migration off GITHUB_TOKEN (Wave B scope covers rotation, not migration).
  • GH_PAT_ADMIN migration to any App — the whole point of admin PATs is operations outside App scope.
  • The n8n workflow-export tooling (out of scope; existing manual export process from #2141 remains).
  • CI runner (GH_TOKEN in GH Actions) — GH provides its own GITHUB_TOKEN env var per-job, no PAT needed.

Plan-PR only. Implementation follows in the child issues (§5.4) after /review-plan iteration.

File-overlap signature: docs/plans/plan-4065-github-pat-autonomous-rotation.md

files_to_change: docs/plans/plan-4065-github-pat-autonomous-rotation.md

Graph View