007-shared-setup-token-claude-auth

ADR 007: Per-Licence setup-token OAuth Tokens for claude-runner Accounts

Status: Accepted 2026-10-01 (owner decision: move to token mode) — plumbing implemented, default off; no host is switched and no token is minted/delivered by the implementing PR. Pilot on lap2 first. Date: 2026-10-01 Relates: ADR 002 (account topology), ADR 006 (WSL worker host class), docs/playbooks/claude-oauth-reauth.md, #4326, #3803, #4854

Context

Topology (confirmed 2026-10-01): two Claude licences, mapped to two Linux accounts — claude-runner (radieu) and claude-runner-2 (ecotrans). bms-4 runs agent sessions on both with the dispatcher load-balancing across them for even usage (scripts/lib/account_routing.py). The same two accounts are wanted on lap2 and windows-dev WSL as additional agent hosts. The Windows host sessions on windows-dev and lap2 themselves are interactive and work on whichever licence is logged in there — they are out of scope and unchanged.

ADR 002 settled on one claude auth login per host per account (copying .credentials.json causes a refresh-token race). With 3 hosts × 2 accounts that is 6 browser logins, 6 refresh crons, and — on WSL, whose instance is unstable (#6185) — frequent re-auth.

Decision

Authorise twice, not per host: mint one claude setup-token token per licence and give each Linux account its own token on every host that has that account.

• claude-runner → token A on bms-4, lap2 WSL, windows-dev WSL; claude-runner-2 → token B on the same.
• Delivered as /etc/p24/claude-oauth-token.<account>.env (CLAUDE_CODE_OAUTH_TOKEN=…, 0600, owned by that account) — a host never holds the other licence’s token for an account it does not run.
• Tokens are not refreshed, so the multi-host refresh race behind the .credentials.json copy ban does not apply to them. Copying these tokens is safe; copying .credentials.json stays forbidden.
• Opt-in per host: claude_shared_oauth_token_enabled: true. Then the refresh cron is removed (role and bms-4.yml) and the role fails unless every account’s file exists, is owned by that account and is 0600. Consumers load the token through one helper, scripts/lib/claude-token-env.sh (Python twin scripts/lib/claude_token.py), which parses the file (a root caller never executes a file another uid can rewrite) and, only when a token is present, unsets ANTHROPIC_API_KEY/ANTHROPIC_AUTH_TOKEN (both outrank the OAuth token). With no token present it is a strict no-op.
• Callers wired: spawn-worker.sh (before the session check, which is skipped in token mode), spawn-infra-task-worker.sh, claude-nightly.sh.j2 (inside the runuser -l command, after .claude-env), run-nightly-triage.sh, run-nightly-design-issues.sh, p24-infra-nightly.sh, queue-analyst.py, run-windows-dev-worker.sh (forwards the token to the container via -e VAR, no value in argv), and login shells via /etc/profile.d (removed again when the flag is off).
• Monitoring: claude-auth-synthetic-check.py and check-sub-usage.py probe with the account’s token (fixes headroom going blind/zero once .credentials.json stops refreshing); an auth failure without rate-limit headers no longer writes a bogus 0% status; the unreadable-token-file case fails instead of falling back to stale creds. claude_runner_token_expires_seconds is derived from the token file’s age (335-day assumed life, conservative).
• lap2 and windows-dev WSL are not in the Ansible inventory; same file layout via their own provisioning, plus the synthetic-check cron must be deployed there by hand (playbook §4).

Consequences

• (+) Two browser authorisations for the whole fleet; WSL restarts no longer force re-auth.
• (−) A token leak or revocation darkens that licence on all hosts at once; each licence needs an expiry calendar entry and an alert before it lapses.
• (−) Tier 1 secrets with fleet-wide blast radius: SOPS, secret-manager role only, narrow recipients.
• (−) Usage/headroom (#7832 — confirmed account-global at source; residual risk is staleness): every host draws on the same two licence quotas. The headroom number is account-global, confirmed from the code: check-sub-usage.py POSTs to https://api.anthropic.com/v1/messages with the account’s own OAuth token and reads the licence-wide rate-limit headers (anthropic-ratelimit-unified-5h-utilization / -7d-utilization, lines ~500–554), which report that licence’s true fleet-wide usage — so every host reading with a given licence’s token sees the same number, not its own local consumption. The dispatcher does not fetch it live, though: each host’s cron writes its own ~/.claude/sub-status.json every ~10 min, and scripts/lib/account_routing.py::account_headroom() reads that host-local file through a 300 s in-memory TTL cache, with a near-limit (headroom ≤ 10%) live-re-read bypass but no file-mtime staleness guard. So the real failure mode is not per-host-vs-global ambiguity but cross-host staleness: between two hosts’ 10-min cron refreshes, both can read a stale “idle” value for the same licence and co-dispatch to it. Proposal (gate 2, not implemented here): (a) add a staleness guard — when sub-status.json’s last_updated/mtime is older than ~2× the cron interval, treat headroom as None (unknown) so an ancient value never ranks a licence as idle; and/or (b) since the source is account-global, converge the hosts on one value by publishing each probe to a shared per-account store (e.g. a Supabase row) and ranking from the most-recent row across hosts, rather than from N independent host-local files. Add before onboarding lap2/WSL to the balancer.
• (−) claude_runner_token_expires_seconds (refresh script) still reads .credentials.json; follow-up.
• Verified 2026-10-01 against https://code.claude.com/docs/en/authentication (“Generate a long-lived token”, “Authentication precedence”):
  • Lifetime: setup-token generates “a one-year OAuth token”; no refresh is documented — expiry needs a calendar entry per licence.
  • Precedence: CLAUDE_CODE_OAUTH_TOKEN (rank 5) beats /login credentials in .credentials.json (rank 7), but is outranked by cloud-provider vars, ANTHROPIC_AUTH_TOKEN, ANTHROPIC_API_KEY and apiKeyHelper. The nightly/spawn scripts therefore unset ANTHROPIC_API_KEY ANTHROPIC_AUTH_TOKEN in token mode.
  • Limits: inference only — no Remote Control sessions, no claude.ai connectors (local MCP works); --bare ignores the token (no script here uses --bare); requires Pro/Max/Team/Enterprise; Desktop/cloud sessions do not read it.
• ToS/concurrency of one licence on many machines: accepted as a monitored risk (ADR 002, reaffirmed 2026-10-01).

Remaining gates before enabling on a production host (not in the implementing PR)

1. secret-manager: mint both tokens; create the narrow SOPS file secrets/claude-oauth.env.sops with keys CLAUDE_OAUTH_TOKEN_CLAUDE_RUNNER + CLAUDE_OAUTH_TOKEN_CLAUDE_RUNNER_2; update the CLAUDE.md SOPS file map. DONE in code (#7832): the .sops.yaml recipient rule is pre-declared (developer + CI AGE_KEY_GHA + lap2-windows, mirroring anthropic-admin) and the secrets-sync.yml delivery jobs exist and are guarded — sync-claude-oauth-bms4 (claude-runner + claude-runner-2 on bms-4) and sync-claude-oauth-vps-i1 (claude-runner on vps-i1; github-runner deliberately excluded as a non-ADR-007 account). A no-op until the SOPS file is minted; per-account placeholder/empty values are skipped (never writes a blank token). Still owned by secret-manager: the actual mint + encrypt of the file, and the CLAUDE.md SOPS-map entry. vps-i1 note: vps-i1 is not in the §Decision host list; the sync job exists so a minted token fans out IF the owner adds vps-i1 to the rollout, and needs the claude-admin-sudo-for-/etc/p24 pre-activation check (playbook §3) first.
2. Headroom (#7832): account-global confirmed (see §Consequences). Still to do: the staleness guard + cross-host convergence proposal in §Consequences — not implemented here.
3. Remaining claude callers (#7832 — DONE/triaged): the self-hosted GitHub workflows (nightly-design-issues, nightly-devops-triage, resource-incident-triage, weekly-supabase-advisor-audit, update-claude-env) and whatsapp-extractor/thread_builder.py now load the helper inline; auto-fix-gh-actions is covered by its runuser -l claude-runner login shell (the /etc/profile.d hook); claude-agent-runner.sh sources the helper. bms4-proxy/session_custody.py is a custody state machine, not a claude invoker (nothing to wire). scripts/tests/test_claude_token_wiring.py now gates CI so no new caller can be added unwired. Residual (sys-admin follow-up): three standalone deployed services run claude in an env they inherit, not via the helper — telegram-claude-bot/claude-session-manager/server.py (tmux session) and the two lint-discovered callers gmail-session-manager/server.py and github-agent/local-listener.py — each needs a systemd EnvironmentFile=-/etc/p24/claude-oauth-token.<account>.env on its unit. Do not enable token mode on a host that runs an un-EnvironmentFile=’d one of these until done.
4. Pilot on lap2, then windows-dev WSL, then bms-4; ADR 002 cleanup once adopted.
5. Compliance: dev_r_services / ops-doc entries for the new credential class once the tokens exist.
6. Residual risk accepted: the token is exported to every child process of a worker (incl. roles denied SOPS/SSH); inference-only scope bounds it. CLAUDE_CODE_SUBPROCESS_ENV_SCRUB not verified. Log-redaction of sk-ant-oat* in worker logs is a hardening follow-up.