p24-infra Docs

gcp-programmatic-management

8 min read

Playbook: Programmatic Google Cloud Management (p24-infra-admin SA)

Goal: let a p24-infra admin agent manage Google Cloud resources programmatically — service accounts, API keys, IAM roles, API enablement, and OAuth redirect URIs — without manual GCP Console work. Backed by a single dedicated service account, p24-infra-admin, whose JSON key lives encrypted in SOPS.

Tracking issue: radieu/p24-infra#2002 Related: docs/secrets-management.md, docs/playbooks/dev-r-services-insert.md, docs/playbooks/credential-rotation-180d.md

Security — read first

  • Never print the SA JSON key, any generated API key, or any SA key file content in chat, commits, or logs. Reference key NAMES only (GCP_SA_KEY_JSON).
  • The SA key is a highly privileged credential — it grants IAM admin, key admin, and project IAM admin across the target project(s). Treat a leak as a P1 incident (docs/playbooks/static-api-key-incident-rotation.md).
  • All generated /tmp/*.json key files MUST be shred -u’d immediately after use — never left on disk, never committed.
  • Secrets live in SOPS+age only. On workers the age key is /home/claude-runner/.age/p24-infra-keys.txt (SOPS_AGE_KEY_FILE).

Deployment target — none (developer/admin-only) — #2944

GCP_SA_KEY_JSON has no server or Vercel deployment target. It is consumed on-demand by Claude admin agents, which decrypt it straight from SOPS into a temp file for gcloud auth activate-service-account (§2) and shred -u it immediately after use. No deployed service, container, or Vercel app reads it, so it is excluded from secrets-sync.yml (!secrets/gcp.env.sops in the push paths-ignore list — do not remove) and is documented as developer-only in .sops.yaml. Treat it like administration.env.sops / bms-servers.env.sops: never synced, always read fresh from SOPS.

Storage note: the key currently lives in the standalone secrets/gcp.env.sops file (not secrets/administration.env.sops). The secrets/gcp.env.sops references throughout this playbook reflect that. gcp.env.sops keeps the full 6-recipient encryption rule so the bms-4 and vps-i1 runner agents can decrypt it at runtime.

1. One-time human setup (≈15 min) — required before any automation

These steps need the GCP Console and a human with Owner/IAM-admin on the GCP project. Claude cannot do them — they bootstrap the very credential the automation depends on.

1.1 Create the service account

GCP project: p24-infra (or whichever GCP project holds shared infra).

Name: p24-infra-admin
ID:   p24-infra-admin

1.2 Assign roles

RoleWhy
roles/iam.serviceAccountAdmincreate/delete service accounts
roles/iam.serviceAccountKeyAdmincreate/rotate SA JSON keys
roles/serviceusage.serviceUsageAdminenable/disable APIs per project
roles/apikeys.admincreate/rotate API keys
roles/resourcemanager.projectIamAdminbind IAM roles on the project
roles/iam.securityAdminmanage IAM policy / audit roles

1.3 Generate the JSON key

Console → IAM & Admin → Service Accounts → p24-infra-admin → Keys → Add Key → JSON → download.

1.4 Add the key to SOPS

The key is added as GCP_SA_KEY_JSON — the entire JSON compacted to a single-line, JSON-escaped string — into secrets/gcp.env.sops (developer/admin-only; never deployed by CI).

On a Linux box with the developer age key (compacting + escaping the downloaded file in one step, without ever printing it):

export SOPS_AGE_KEY_FILE="$HOME/.age/p24-infra-keys.txt"
 
# Build the single-line value silently (jq -c compacts; the value never hits the terminal).
VAL=$(jq -c . /path/to/downloaded-key.json)
 
# Append the line inside sops edit (opens $EDITOR on the decrypted buffer):
sops secrets/gcp.env.sops
#   add a line:  GCP_SA_KEY_JSON=<paste $VAL>
# save + exit — sops re-encrypts automatically.
 
unset VAL
shred -u /path/to/downloaded-key.json

Windows note: if editing SOPS on Windows, follow the CRLF-safe procedure in CLAUDE.md ([System.IO.File]::WriteAllText with UTF8Encoding($false) — never >, Out-File, or Set-Content). See docs/playbooks/sops-windows-crlf.md.

Verify the key is present (NAME only, never the value):

sops -d --input-type dotenv --output-type dotenv secrets/gcp.env.sops \
  | grep -q '^GCP_SA_KEY_JSON=' && echo "GCP_SA_KEY_JSON: exists" || echo "missing"

Commit and push:

git add secrets/gcp.env.sops
git commit -m "chore: add GCP_SA_KEY_JSON to gcp secrets"
git push
# NOTE: gcp.env.sops is developer/admin-only and is EXCLUDED from secrets-sync.yml — CI does NOT
# distribute this value to any server. Admin agents read it on-demand from SOPS (§2). See #2944.

1.5 Signal completion

Post a comment on the tracking issue (radieu/p24-infra#2002) when done. A Claude session then runs §3 (verification), §4 (compliance row), and the automation in §5 on demand.

2. Authenticate as p24-infra-admin (Claude, after §1)

Load the key from SOPS into a temp file, activate it for gcloud, then shred the file. The key value is never echoed.

export SOPS_AGE_KEY_FILE="$HOME/.age/p24-infra-keys.txt"
KEYFILE=$(mktemp /tmp/gcp-sa-XXXXXX.json)
chmod 600 "$KEYFILE"
 
# Decrypt only this one key into the temp file (value never printed):
sops -d --input-type dotenv --output-type dotenv secrets/gcp.env.sops \
  | grep '^GCP_SA_KEY_JSON=' | cut -d= -f2- > "$KEYFILE"
 
gcloud auth activate-service-account --key-file="$KEYFILE"
 
# ... run gcloud commands ...
 
gcloud auth revoke --all >/dev/null 2>&1 || true
shred -u "$KEYFILE"

If gcloud is not installed on the worker, install the Google Cloud CLI first (docs/playbooks/ — or curl https://sdk.cloud.google.com | bash) and re-auth.

3. Verify (AC: gcloud projects list works)

After §2 activation:

gcloud projects list   # must return the accessible projects without an auth error

A non-empty list with no PERMISSION_DENIED confirms the SA + roles are correct. Record the outcome on the tracking issue (“verified — gcloud projects list returns N projects”); never paste project IDs that are sensitive.

4. Compliance row (AC: dev_r_services)

Once GCP_SA_KEY_JSON exists, register the credential in dev_r_services. Follow the trigger-enforced-column rules in docs/playbooks/dev-r-services-insert.md (note: no unique constraint on service_name, so use the WHERE NOT EXISTS guard, not ON CONFLICT):

Always set the rotation-tracking columns (criticality, rotation_freq, last_rotated, next_due, auto_rotate). Omitting them leaves next_due/criticality NULL, which makes the credential-exporter emit its 999 “no due date” sentinel and criticality=unknown — a false CredentialRotationCritical alert (#2171). This is a manually-rotated key (no self-rotate API), so auto_rotate=false on a 180-day cadence, same as CF_GLOBAL_API_KEY.

DO $$
BEGIN
  IF NOT EXISTS (SELECT 1 FROM dev_r_services WHERE service_name = 'gcp-sa-p24-infra-admin') THEN
    INSERT INTO dev_r_services (
      service_name,
      project_id,   project_name,
      office_id,    ws_id,    app_id,
      service_type,
      element_type,
      status,
      owner,
      criticality,
      rotation_freq,
      last_rotated,
      next_due,
      auto_rotate,
      compliance_workbook,
      workbook_url,
      compliance_notes
    ) VALUES (
      'gcp-sa-p24-infra-admin',
      'p24-infra', 'p24-infra',
      'p24-devops', '99', 'et-app',
      'secret',
      'credential',
      'active',
      'radieu',
      'high',
      '180 days',
      CURRENT_DATE,
      CURRENT_DATE + interval '180 days',
      false,
      'yes',
      'docs/playbooks/gcp-programmatic-management.md',
      'GCP service account for programmatic IAM/API-key/SA/OAuth management; JSON key in secrets/gcp.env.sops as GCP_SA_KEY_JSON (developer/admin-only, not CI-deployed)'
    );
  ELSE
    UPDATE dev_r_services SET
      service_type        = 'secret',
      element_type        = 'credential',
      status              = 'active',
      criticality         = COALESCE(criticality, 'high'),
      rotation_freq       = COALESCE(rotation_freq, '180 days'),
      last_rotated        = COALESCE(last_rotated, CURRENT_DATE),
      next_due            = COALESCE(next_due, CURRENT_DATE + interval '180 days'),
      auto_rotate         = false,
      compliance_workbook = 'yes',
      workbook_url        = 'docs/playbooks/gcp-programmatic-management.md'
    WHERE service_name = 'gcp-sa-p24-infra-admin';
  END IF;
END $$;

Verify exactly one row:

SELECT service_name, service_type, element_type, compliance_workbook,
       criticality, rotation_freq, next_due, auto_rotate
FROM dev_r_services WHERE service_name = 'gcp-sa-p24-infra-admin';

criticality, rotation_freq, and next_due must all be non-NULL — otherwise the credential-exporter will flag a false CredentialRotationCritical (999d / unknown).

5. Ongoing automation (after §1)

All commands assume the SA is active (§2). Run inside the auth/shred wrapper.

5.1 OAuth redirect URIs — single shared OAuth app

Architecture: one OAuth app (p24-platform, created once by a human) serves all projects. The redirect URI is always:

https://p24-auth.radieu.workers.dev/callback

New CF Workers / projects never need a new OAuth client — they reuse the shared GOOGLE_CLIENT_ID (add it to the project’s secrets/*.env.sops, no new GCP OAuth app). Add a redirect URI to the existing app only if a genuinely new callback host is introduced:

gcloud alpha iap oauth-clients update <oauth-client> \
  --add-redirect-uris="https://p24-auth.radieu.workers.dev/callback"

5.2 New-project bootstrap (enable API + dedicated SA + key)

Example: a new project needs the Google Sheets API and its own worker SA.

PROJECT=<project>
NAME=<name>     # short slug, e.g. "sheets-sync"
 
gcloud services enable sheets.googleapis.com --project="$PROJECT"
 
gcloud iam service-accounts create "p24-${NAME}-worker" --project="$PROJECT" \
  --display-name="p24 ${NAME} worker"
 
KEYFILE=$(mktemp /tmp/gcp-newsa-XXXXXX.json); chmod 600 "$KEYFILE"
gcloud iam service-accounts keys create "$KEYFILE" \
  --iam-account="p24-${NAME}-worker@${PROJECT}.iam.gserviceaccount.com"
 
# Add the new key to the appropriate secrets/*.env.sops (NOT administration unless infra-shared),
# compacting with jq -c as in §1.4. Then shred:
shred -u "$KEYFILE"

Open a PR with the SOPS update and a dev_r_services row for the new SA (§4 pattern) plus an ops note. Never commit the plaintext key.

5.3 SA key + API key rotation

On schedule (see docs/playbooks/credential-rotation-180d.md) or on demand:

# List existing keys for an SA (key IDs only — no private material printed):
gcloud iam service-accounts keys list --iam-account="<sa-email>"
 
# Create a new key, update SOPS (§1.4 / §5.2), confirm the new value is live, THEN delete the old:
gcloud iam service-accounts keys delete <OLD_KEY_ID> --iam-account="<sa-email>"

Always: create new → distribute via SOPS+CI → verify live → revoke old. Log the rotation in docs/secrets-rotation-log.md.

5.4 IAM audit (detect over-privileged roles)

gcloud projects get-iam-policy <project> --format=json \
  | jq '.bindings[] | {role, members}'

Flag any member with broad roles (roles/owner, roles/editor) that should be scoped down. File a GH issue for any finding.

5.5 Enable / disable APIs per project

gcloud services list --enabled --project=<project>          # audit
gcloud services enable  <api>.googleapis.com --project=<project>
gcloud services disable <api>.googleapis.com --project=<project>   # confirm no dependents first

6. Error reporting (mandatory)

Any automation script built on this playbook must, on failure, send a Discord embed via P24_DISCORD_INFRA_SCRIPTS_ERRORS_WEBHOOK_URL and create a GH issue in radieu/p24-infra with label bug (see CLAUDE.md §Error Notification Standard). Never silently swallow a failure.

7. Checklist

  • §1 human setup done; GCP_SA_KEY_JSON present in secrets/gcp.env.sops
  • §3 gcloud projects list verified
  • §4 dev_r_services row gcp-sa-p24-infra-admin present with non-NULL criticality/rotation_freq/next_due
  • Every SA key file shred -u’d after use; no key value ever printed
  • Rotations logged in docs/secrets-rotation-log.md

Graph View