p24-infra Docs

2650-wasabi-api-key-rotation-feasibility

3 min read

Feasibility — Automatic Wasabi API Key Rotation

Issue: #2650 · Status: Design (feasibility research) · Batch: K (#3338)

Can Wasabi user API keys (access key + secret) be rotated programmatically? Yes — Wasabi exposes an IAM-compatible control API, and a user may hold two access keys at once, which enables zero-downtime rotation. This doc records the finding and the rotation design; implementation is separate work.

1. Finding

Wasabi’s control plane is S3-IAM-compatible. Access keys are managed with the same CreateAccessKey / DeleteAccessKey / ListAccessKeys calls as AWS IAM, against Wasabi’s IAM endpoint (region-scoped service URL — see the Wasabi service-URLs article referenced on the issue).

Key facts:

  • Two-key limit per user → the standard AWS-style overlap rotation works: create key #2, roll it out to all consumers, verify, then delete key #1.
  • Requires an admin/control credential with IAM permissions — this is WASABI_ADMIN in secrets/administration.env.sops (developer-only, never CI-deployed). Rotation of a service key is therefore driven by an admin credential, not the service key itself.
  • We already operate a related pattern: wasabi-iam-rotator (Node.js on Vercel, see docs//CLAUDE.md service table) rotates S3 IAM credentials for the p24-infra bucket. That service is the natural home/reference implementation for this capability.

2. Rotation design (overlap / two-key)

1. ListAccessKeys(user)              → assert count == 1 (else abort: manual cleanup needed)
2. CreateAccessKey(user)             → key#2 (new AccessKeyId + SecretAccessKey)
3. Distribute key#2:
     - SOPS: update the owning secrets/*.env.sops (secret-manager session)
     - secrets-sync.yml → live envs + containers + Vercel
4. Verify key#2 works (HEAD bucket / list objects with new creds)
5. DeleteAccessKey(user, key#1)      → old key revoked, back to count == 1
6. Log rotation in docs/secrets-rotation-log.md

Failure at any step before 5 is safe — key#1 still works, so no outage. Only step 5 is destructive.

3. Which keys this applies to

Key (name only)SOPS fileNotes
Wasabi p24-infra (monitoring/backups)secrets/monitoring.env.sopsprimary rotation target
WASABI_ADMINsecrets/administration.env.sopsthe driver — cannot self-rotate via this flow; manual

4. Recommendation & effort

  • Feasible and low-risk thanks to the two-key overlap. Recommend implementing as a scheduled rotation inside/alongside wasabi-iam-rotator, or a new scripts/ cron following the Error Notification Standard (Discord + GH issue on failure).
  • Effort: ~S/M — the IAM calls are standard; most work is safe distribution + verification wiring.
  • Constraint: all SOPS writes go through a secret-manager session — this doc/issue is dev-scope and does not touch SOPS.

5. Out of scope for this doc

No key is created, distributed, or deleted by this issue. Deliverable is the feasibility verdict + rotation design.

Design-only deliverable. Related: wasabi-iam-rotator service, docs/password-rotation-procedures.md, docs/secrets-rotation-log.md.

Graph View