lap2 — radieu (local, second dev workstation)

Environment ID: lap2 Type: dev + worker host Status: active (human dev role) · worker host ACTIVE (dev_r_server_capacity.enabled=true, id 17). ADR-006 Phase 1 complete and the heartbeat + keep-WSL tasks are provided (#7664, scripts/lap2-heartbeat-task.ps1 / scripts/lap2-keep-wsl-up.ps1). Remaining human steps: register the two Task Scheduler tasks on the host (runbook: docs/playbooks/server-heartbeat.md §“Setup on lap2”) and run the one-time activation soak — dispatch one light test job and confirm it ends done with the ai-dev-* identity; set enabled=false on any failure.

Purpose

Second human developer machine (Dell laptop), used the same way as dev-workstation — code authoring, SSH access to all servers, GitHub operations, orchestrating VPS-based Claude Code agents. Referenced throughout the repo (SOPS catch-all rule comments, root_trusted_authorized_keys entries, several PR bodies) since #6890 even though this doc itself didn’t exist until now — created 2026-09-15 while closing out gaps found during the #6857 runbook.

Also a WSL2 Docker ephemeral-worker host (ADR-006), the second host in that worker class after windows-dev. Unlike windows-dev (a distinct doc for the WSL2 persona of dev-workstation), lap2’s Windows host and its WSL2 worker host are documented together here since it’s the same physical machine and the same session context switches between the two roles. See §WSL2 Worker Host below for the worker-host-specific hardware, capacity, dispatch path and secrets model — those are governed by ADR-006, not by the rest of this doc’s §Hardware/§Access/§Secrets Access (which describe the human-workstation role only).

Hardware

FieldValue
OSWindows 11 Pro 10.0.26200
ShellBash (Git Bash) is the tool actually used in this environment — unlike dev-workstation, do not assume PowerShell-only; check the session’s own tool availability first
UserRadek (Windows account) — not konar, the account on the original dev-workstation machine
Repo pathC:\devWork\p24-infra — not C:\code_2026\p24-infra

Software Stack

ToolPurpose
Git + GitHub CLI (gh)Version control and GitHub API operations
SOPS + ageSecret decryption (C:\Users\Radek\.age\p24-infra-keys.txt)
SSH (C:\Users\Radek\.ssh\id_ed25519, comment radieu@lap2)Access to all VPS and BMS servers
Python 3.xScripts, SOPS operations

Access

  • All VPS and BMS servers via ssh -i C:\Users\Radek\.ssh\id_ed25519 root@<ip> — key added to root_trusted_authorized_keys in #6936 (2026-09-09), applied fleet-wide via ansible-playbook ansible/playbooks/ssh-authorized-keys.yml.
  • SOPS: onboarded onto the catch-all recipient rule in #6890 (2026-09-06) — can decrypt every secrets/*.env.sops file that isn’t narrow-scoped to dev-workstation’s specific recipient list.
  • GitHub: radieu account with full repo access.

Constraints

  • Do not assume the C:\Users\konar\... / C:\code_2026\p24-infra paths hardcoded in various scripts and in CLAUDE.md’s older examples — those are dev-workstation-specific. If a script or doc hardcodes one of those paths and fails here, that’s the known cross-workstation staleness pattern (see #7067 for one instance, env-lock.ps1) — verify the actual local path/profile rather than editing the shared script to match only this machine.
  • Same SOPS write discipline as dev-workstation: UTF8 no-BOM writes only, canary-decrypt before commit — see CLAUDE.md §Safe secret handling.
  • This machine and dev-workstation are used concurrently by different sessions against the same shared GitHub remote — do not assume a local, non-worktree checkout (C:\devWork\p24-infra itself) reflects the latest origin/main; another session may have pushed since you last fetched. For any live/production operation that depends on the current value of a file (especially secrets/*.env.sops), git fetch origin main and verify before acting, or use an isolated git worktree (see CLAUDE.md §Local Interactive Worktrees) rather than the shared checkout.

gmail-tools (local repo + OAuth credentials)

gmail-tools (the separate repo CLAUDE.md’s §Email section refers to) is checked out here at C:\devWork\gmail-tools — not C:\code_2026\gmail-tools, the dev-workstation-specific path hardcoded in that repo’s own docs (docs/playbooks/gmail-tools-oauth-secret-sync.md etc.) and in some of its scripts’ comments. Same cross-workstation staleness pattern as the rest of this doc’s §Constraints — verify the local path rather than trusting a hardcoded one.

This host had no OAuth credential files at all until 2026-09-29 (~/.gmail-mcp/ didn’t exist — gmail-tools/credentials/ isn’t used here either, this repo’s checkout has no such directory). Materialized both files that lib/credentials-path.js expects, from secrets/gmail-tools.env.sops (never printed, one-shot write via a scratch Python script — see #7009 session history for the exact commands if needed again):

FileSource SOPS keys
C:\Users\Radek\.gmail-mcp\gcp-oauth.keys.jsonGMAIL_TOOLS_GCP_OAUTH_CLIENT_ID + GMAIL_TOOLS_GCP_OAUTH_CLIENT_SECRET, wrapped in the {"installed": {...}} shape auth.js/gmail-query.js expect (raw.installed || raw.web)
C:\Users\Radek\.gmail-mcp\token.jsonGMAIL_TOOLS_OAUTH_TOKEN — already the exact JSON shape (access_token/refresh_token/scope/token_type/expiry_date) oAuth2Client.setCredentials() expects, written as-is

Verified working same-day: node scripts/gmail-query.js --query "in:inbox newer_than:1d" --max 1 returned real results, and node scripts/send-digest.js sent a real test message.

These are local dev-machine files, not SOPS — per gmail-tools-oauth-secret-sync.md’s §Prevention, nothing auto-syncs them; a GCP-console-triggered client rotation (outside the normal SOPS-edit flow) will silently desync them until gmail-query.js starts failing with invalid_client/invalid_grant. If that happens here, follow that playbook’s resync procedure but substitute this doc’s paths (C:\devWork\gmail-tools, C:\Users\Radek\.gmail-mcp\) for its C:\code_2026\... / konar ones.

Known Issues

  • scripts/env-lock.ps1 and possibly other scripts hardcode the dev-workstation age-key path — see #7067.
  • Prior to #6936 (2026-09-09) this machine had no SSH access to bms-2/bms-3/bms-4 at all — Permission denied (publickey) on any SSH attempt. Fixed, but any playbook/doc written before that date may still describe this machine as unreachable.

Secrets Access

Secret storeAccess methodCan decrypt SOPS?
Catch-all-rule secrets/*.env.sops filesage key at C:\Users\Radek\.age\p24-infra-keys.txtYes — same catch-all recipient set as dev-workstation, added #6890
Narrow-scoped SOPS files (administration.env.sops, role-*.env.sops, etc.)Not a recipient unless explicitly addedNo, by design — confirmed via .sops.yaml, administration.env.sops’s creation_rule lists exactly 2 recipients (dev-workstation, lap2-windows’s peer, neither is lap2’s worker-container key)
GitHub Actions secretsNot accessible from this workstation directlyN/A

WSL2 Worker Host (ADR-006 Phase 1)

The second host in the WSL2 worker-host class, onboarded via the exact 9-item checklist in ADR-006 (the same pattern generalized from windows-dev). This section documents the worker-host role only — the Docker container’s secrets model is independent from the human-workstation SOPS access in §Secrets Access above.

Hardware (WSL2 guest)

FieldValue
Labellap2 (dev_r_server_capacity.server_label)
Host OSWindows 11 Pro (this machine — see §Hardware above), hostname DESKTOP-FETFG2J
Guest OSUbuntu-26.04 (WSL2, systemd enabled), kernel 6.18.33.2-microsoft-standard-WSL2, WSL subsystem 2.7.12.0
CPU12 vCPUs (os_vcpus)
RAM24 GB WSL2 allocation (os_ram_gb; usable worker budget ≈ 19 GB after the 80 % cap)
Disk~1007 GB total, ~945 GB free
InboundCF Tunnel: lap2-ssh.zintegrowana.online (no public IP; two Access policies — owner-only email/OTP + a non_identity Service Token policy for the dispatcher, PR #7071, same pattern as the windows-dev #6740 incident fix)

Capacity (dev_r_server_capacity, id 17)

FieldValueWhy
enabledtrueADR-006 Phase 1 complete; heartbeat mechanism provided (#7664). The 30-min live soak is a one-time human check run after the Task Scheduler tasks are registered — flip to false if it fails
max_workers_prime / max_workers_night5 / 7steady-state concurrent-worker caps once the soak proved headroom (superseded the 1/1/0 MVP posture windows-dev shipped with)
max_weight_prime / max_weight_nightheavy / heavyheavy enabled after the soak. playwright is NOT yet enabled — gated on the per-server playwright cap (#7718) and a headless-Chromium smoke test; see §Playwright concurrency cap below
max_concurrent_heavyNULL (uncapped)#7718 — per-server cap on concurrently active heavy/playwright/orchestrator workers. NULL ⇒ dispatcher default: on lap2 (a non-bms host) that is uncapped, so heavy is gated only by RAM + max_workers_*
max_concurrent_playwrightNULL → target 1#7718 — playwright-only cap. NULL ⇒ uncapped; set to 1 when playwright is enabled so lap2 runs exactly one playwright worker at a time (6 GB each would otherwise let ~3 run inside the ~19 GB budget)
weight_ram_gbsubagent=1, light=3, heavy=6, playwright=6, orchestrator=12Docker -m hard cap per weight tier — lap2-specific, deliberately not identical to windows-dev.md’s tiers (light=2, heavy=7, etc.)
os_ram_gb / reserved_ram_gb24 / 0WSL2 allocation; usable worker budget = (24 - 0) * 0.8 ≈ 19 GB (the 80 % cap the dispatcher + Grafana share)
emergency_max_workers16hard cap, not the steady-state cap
claude_accountsclaude-runner, claude-runner-2independently logged in via claude auth login in /root/.claude-configs/claude-runner{,-2} (own device login, ADR-002 — never copied from another host). radieu deliberately excluded, same reasoning as windows-dev: it’s the developer’s own daily-driver account and would compete with interactive use on this same machine

Playwright concurrency cap (#7718)

The dispatcher reads two nullable dev_r_server_capacity columns as per-server concurrency ceilings (added by migration 20261010093108_server_capacity_concurrency_caps_7718.sql):

  • max_concurrent_heavy — generalises the old hardcoded BMS4_MAX_CONCURRENT_HEAVY / BMS3_MAX_CONCURRENT_HEAVY = 1 constants (heavy/playwright/orchestrator combined). NULL keeps the historical default: 1 on bms-4/bms-3, uncapped on every other host (lap2 included).
  • max_concurrent_playwright — a playwright-only ceiling. NULL ⇒ uncapped. When the cap binds, the dispatcher stops offering playwright rows for the rest of the cycle but keeps dispatching lighter jobs into lap2’s remaining slots (it does not idle the host), so a queued playwright job never blocks the light/heavy jobs behind it.

Why lap2 needs it before enabling playwright. The p24-windows-dev-worker:latest image already ships Playwright + Chromium (PLAYWRIGHT_BROWSERS_PATH=/opt/ms-playwright). But with playwright=6 GB/worker and a ~19 GB usable budget, up to 3 playwright workers could run at once if max_weight_* were raised to playwright with no cap. max_concurrent_playwright=1 makes lap2 run exactly one playwright worker at a time.

Activation (post-merge, #7718 deliverables 5–6, human/secret-manager + sys-admin):

  1. Run a Playwright headless-Chromium smoke test inside the lap2 worker container and confirm it launches (deliverable 6 — a sys-admin/infra-task step on the host).
  2. Set max_weight_prime / max_weight_night = playwright and max_concurrent_playwright=1 on the lap2 dev_r_server_capacity row (id 17). Then the enabled-true row’s heartbeat picks up the new weight ceiling on the next dispatcher cycle.

Dispatch path

Identical shape to windows-dev’s (see windows-dev.md for the full exit-code translation contract — ADR-006 reuses it unchanged):

bms-4 dispatcher (queue-dispatcher-loop.py)
  → CF Tunnel (lap2-ssh.zintegrowana.online)
  → scripts/run-windows-dev-worker.sh  ISSUE JOB_TYPE WEIGHT ...
      → docker run --memory <G> --memory-swap <G> --cpus <n>  p24-windows-dev-worker  <args>
          (built locally on lap2 with --build-arg SERVER_LABEL=lap2, PR #7094 — the image was
          previously hardcoded to "windows-dev" and would have silently misattributed every lap2
          job in worker IDs/audit trails)
          → entrypoint.sh → spawn-worker.sh  (WORKER_ISOLATION=docker, foreground, no systemd)
      → translate outcome → dispatcher's five-code contract

Secrets access (container-mount level, ADR-005 §8 — same model as windows-dev)

  • secret-manager-role containers only get the SOPS age key mounted in.
  • sys-admin / sys-security get read-only credential access for verification (not yet wired on the docker-run path — a follow-up to #7104).
  • dev-coder and every other role get no SOPS/age-key access at all — never mounted.
  • Every research/investigation subagent still carries the mandatory #5544 “No SOPS decryption” clause inside a container, identically to any other worker host.

How it is enforced (#7104). scripts/run-windows-dev-worker.sh (the shared host wrapper for windows-dev and lap2) reads the ROLE positional arg ($7) and read-only-mounts the age key only for secret-manager: -v $WORKER_AGE_KEY_FILE:/home/claude/.age/p24-infra-keys.txt:ro. WORKER_AGE_KEY_FILE is a host env var set at provisioning — lap2’s own automation age-key path (distinct from dev-workstation’s personal key). A secret-manager job dispatched with it unset or pointing at a missing file fails closed (wrapper exits 1) instead of running SOPS-blind.

GH_TOKEN credential model for Docker workers (#7897)

Workers inside the Docker container resolve GH_TOKEN through a 3-step chain (see scripts/spawn-worker.sh §GH_TOKEN):

1. env var set by caller (run-windows-dev-worker.sh → docker run -e GH_TOKEN=...)
2. /usr/local/bin/gh-app-token  [NOT installed in the worker image — chain skips this]
3. /opt/p24-infra/${SERVER_NODE}/.env  [absent inside the container at runtime]

**Before 7897, GH_TOKEN was not included in the env_exports string built by queue-dispatcher-loop.py’s spawn_worker(). The dispatcher loads the token from EnvironmentFile=/opt/p24-infra/bms-4/worker-queue.env (systemd), but only variables explicitly enumerated in env_exports reach the remote shell where run-windows-dev-worker.sh runs. Steps 2 and 3 are both unavailable inside the container → all Docker-based workers received an empty token → gh calls → 401 → git clone exit 6.

Fix (#7897): spawn_worker() in queue-dispatcher-loop.py now appends GH_TOKEN when the dispatcher’s own env has a non-empty value:

if GH_TOKEN:
    env_exports += f" GH_TOKEN={shlex.quote(GH_TOKEN)}"

The token value comes from secrets/worker-queue.env.sops → deployed to /opt/p24-infra/bms-4/worker-queue.env by secrets-sync.yml.

Related gap — the same root cause applied to bms-4 bare-host workers, but those recovered via gh-app-token (step 2). The fix makes the forwarding explicit on every path.

Known gap (SUPA_HB_KEY): the heartbeat script (lap2-heartbeat-task.ps1) resolves its Supabase service-key at Windows Task Scheduler level via Invoke-WithSopsSecret — completely outside the Docker worker dispatch path. It is unaffected by this fix and does not depend on GH_TOKEN.

Heartbeat + keep-alive (#7664)

Two Windows Task Scheduler tasks keep lap2 dispatchable (register per docs/playbooks/server-heartbeat.md §“Setup on lap2”):

TaskScriptCadencePurpose
P24-Lap2-Heartbeatscripts/lap2-heartbeat-task.ps1 → scripts/server-heartbeat.ps1every 2 minPATCH last_heartbeat + current_workers to dev_r_server_capacity (id 17); pushes p24_server_heartbeat_http_status to vps-i1 best-effort
P24-Lap2-KeepWSLscripts/lap2-keep-wsl-up.ps1at logon + every 5 minkeep the Ubuntu-26.04 VM alive via a named p24-wsl-keepalive (sleep infinity)
  • Secrets (SUPABASE_URL / SUPABASE_SERVICE_KEY) are decrypted on-demand from secrets/monitoring.env.sops via the SOPS broker Invoke-WithSopsSecret — the lap2-windows age key is already a catch-all recipient (#6890/#6909), so no secret-manager delivery is needed. No credential is ever written to a file or a task argument.
  • current_workers counts running Docker worker containers inside the WSL2 guest (0 if WSL is down).
  • Sleep/lock caveat: a suspended laptop stops the heartbeat → server_status goes degraded/down; the dispatcher tolerates it (a down host simply claims nothing). Disabling sleep on AC power is a manual, human-reviewed Windows power-settings change, not done by any script.

Pinning a job to lap2 (#7844)

server_preference='lap2' is a valid value in dev_r_worker_queue since migration 20261013150045_worker_queue_server_preference_lap2_7844.sql (the original CHECK constraint only allowed vps-i1/bms-4). Use it to route a specific job exclusively to this host:

-- Insert a pinned light job directly via Supabase SQL / REST:
INSERT INTO dev_r_worker_queue
  (status, job_type, priority, issue_number, github_issue_number, repo, server_preference)
VALUES
  ('queued', 'dev-issue', 50, <issue_number>, <issue_number>, 'radieu/p24-infra', 'lap2');

Or via the REST endpoint (e.g. from scripts/queue-dispatcher-loop.py helpers):

curl -sf -X POST "${SUPA_HB_URL}/rest/v1/dev_r_worker_queue" \
  -H "apikey: ${KEY}" -H "Authorization: Bearer ${KEY}" \
  -H "Content-Type: application/json" -H "Prefer: return=minimal" \
  -d '{"status":"queued","job_type":"dev-issue","priority":50,
       "issue_number":<N>,"github_issue_number":<N>,
       "repo":"radieu/p24-infra","server_preference":"lap2"}'

How the dispatcher claims it. The claim_job PostgREST filter is: or(server_preference.is.null,server_preference.eq.<server_label>). A lap2 dispatcher run claims only rows where server_preference = 'lap2' or server_preference IS NULL. bms-4 skips rows pinned to lap2 (it does not match server_preference.eq.bms-4).

Canary verification. After inserting a pinned row, confirm end-to-end:

  1. Dispatcher log (bms-4): job row status='claimed', claimed_by starts with the lap2 CF Tunnel session.
  2. lap2: docker ps shows a running p24-windows-dev-worker container.
  3. On completion: row status='done', worker_env.host='lap2'.
  4. Peak memory: /var/log/p24-worker-mem.csv on lap2 (service p24-worker-mem-sampler).

If it fails with spawn_failures+timeout (as row 12541 did), capture the dispatcher’s error_message column and the CF Tunnel access log on lap2 to distinguish a tunnel vs. docker-run failure.

OAuth token refresh cron (WSL2 guest) (#7965)

lap2’s Docker worker containers authenticate by mounting a credential dir from the WSL2 guest host (/root/.claude-configs/<account>/) — not from a Linux user’s ~/.claude/. The standard Ansible-managed claude-token-refresh cron (from ansible/roles/claude-runner/templates/claude-token-refresh.j2) hardcodes /home/<user>/.claude/ and does not apply here. The refresh must be set up manually in the WSL2 Ubuntu guest.

What’s needed

Two cron entries per account in /etc/cron.d/claude-token-refresh on the WSL2 guest (Ubuntu):

SHELL=/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

# claude-runner (radieu account)
*/15 * * * * root python3 /opt/p24-infra/scripts/refresh-claude-token.py \
  --user claude-runner \
  --config-dir /root/.claude-configs/claude-runner \
  --uid <uid-of-claude-in-container> \
  >> /var/log/claude-token-refresh.log 2>&1

# claude-runner-2 (ecotrans account)
*/15 * * * * root python3 /opt/p24-infra/scripts/refresh-claude-token.py \
  --user claude-runner-2 \
  --config-dir /root/.claude-configs/claude-runner-2 \
  --uid <uid-of-claude-in-container> \
  >> /var/log/claude-token-refresh.log 2>&1

Finding <uid-of-claude-in-container>: the claude user in the worker image is created by useradd -m -s /bin/bash claude (no explicit --uid, see docker/windows-dev-worker/Dockerfile). Run docker run --rm p24-windows-dev-worker id -u claude to get the uid from the built image (typically 1000 on a fresh Ubuntu base).

Why --uid matters: refresh-claude-token.py runs as root on the WSL2 guest and writes the refreshed .credentials.json back to /root/.claude-configs/<account>/. The container mounts this dir at /home/claude/.claude. Without --uid <container-uid>, the refreshed file is owned by root (uid 0) and the non-root claude user inside the container cannot read it. --uid chowns the refreshed file to the container’s claude uid after each write.

Subscription usage gauge (for ClaudeRunnerTokenExpired alert coverage): the same --config-dir flag now exists on check-sub-usage.py (#7965). Add it to /etc/cron.d/claude-sub-usage (or the equivalent) on the WSL2 guest:

# claude-runner
*/10 * * * * root CLAUDE_SUB_MACHINE=lap2 \
  python3 /opt/p24-infra/scripts/check-sub-usage.py \
  --user claude-runner \
  --config-dir /root/.claude-configs/claude-runner \
  >> /var/log/claude-sub-usage.log 2>&1

# claude-runner-2
*/10 * * * * root CLAUDE_SUB_MACHINE=lap2 \
  python3 /opt/p24-infra/scripts/check-sub-usage.py \
  --user claude-runner-2 \
  --config-dir /root/.claude-configs/claude-runner-2 \
  >> /var/log/claude-sub-usage.log 2>&1

CLAUDE_SUB_MACHINE=lap2 sets the machine= label on the Prometheus gauges so ClaudeRunnerTokenExpired / ClaudeRunnerTokenExpiringSoon fire for machine=lap2 (they scope to enabled hosts via p24_server_max_workers_prime, which already includes lap2 — id 17, enabled=true). Without this cron the only OAuth alerting for lap2 is the reactive ClaudeWorkerOAuthSpawnFailures (fires after a job fails), not the proactive expiry-based alerts.

Also note: run-windows-dev-worker.sh now performs a pre-spawn auth check (#7965): if the access token is already expired and there is no refresh token, it exits 7 (oauth_expired) immediately rather than burning ~80 s in a doomed container. This is a safeguard, not a substitute for the refresh cron — the goal is to keep tokens alive, not just fail faster when they die.

Real auth probe (p4_claude_auth_valid gauge, ClaudeAuthSyntheticFailing alert): the same --config-base and --machine flags now exist on claude-auth-synthetic-check.py (#7965). Add it to /etc/cron.d/claude-auth-synthetic-check (create the file if absent) on the WSL2 guest:

SHELL=/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

*/15 * * * * root python3 /opt/p24-infra/scripts/claude-auth-synthetic-check.py \
  --config-base /root/.claude-configs \
  --machine lap2 \
  >> /var/log/claude-auth-synthetic.log 2>&1

This single cron entry iterates all accounts whose subdir exists under /root/.claude-configs/ (claude-runner, claude-runner-2) and pushes p4_claude_auth_valid{machine="lap2",account="…"} to the vps-i1 Pushgateway per account — the same gauge the ClaudeAuthSyntheticFailing / ClaudeAuthSyntheticStale alerts watch. Without this cron, a dead token on lap2 is caught only reactively (after a job fails) rather than proactively (within 30 min of the first failure).

Verification

After installing all three cron files and waiting one cycle (≤15 min), confirm:

# On the WSL2 guest
tail -20 /var/log/claude-token-refresh.log        # should show "no-op" or "refreshed"
tail -20 /var/log/claude-sub-usage.log             # should show usage=X.X%
tail -20 /var/log/claude-auth-synthetic.log        # should show "Auth VALID for radieu/ecotrans on lap2"
 
# On vps-i1 (Pushgateway), verify gauges are arriving:
ssh root@217.154.82.162 "curl -s http://localhost:9091/metrics | grep claude_runner_token_expires_seconds | grep lap2"
ssh root@217.154.82.162 "curl -s http://localhost:9091/metrics | grep p4_claude_auth_valid | grep lap2"