p24-infra Docs

windows-dev

8 min read

windows-dev — WSL2 Docker Ephemeral-Worker Node

Status update (2026-10-14): LIVE — enabled=true, scaled for 24/7 idle-most-of-the-time operation. #6185 (WSL2 instability) and 4728 (bms-3 RAM-weight bug, the fleet-concurrency sequencing guard) are both CLOSED. On-host prerequisites verified live: Docker 29.7.2 native in WSL2 Ubuntu, systemd + cloudflared active, per-account claude auth login done for claude-runner/claude-runner-2, claude-token-refresh + claude-sub-usage crons installed (mirroring lap2’s #7965 pattern — previously missing here). Capacity raised from the MVP 1/1 light-only posture to max_workers_prime=4 / max_workers_night=8, max_weight_prime=max_weight_night=heavy — the owner’s workstation sits on 24/7 but is rarely interactively used, so night capacity now exceeds day capacity (opposite of the original conservative default). #6740 (CF Access Service Token) and #7980 (SSH auth) both resolved and verified end-to-end, same day. The CF Access service token was already deployed (CF_ACCESS_WINDOWS_DEV_CLIENT_ID/_SECRET in secrets/n8n-bms4.env.sops, synced to bms-4). The real blocker was dev_r_server_capacity.ssh_user='claude' — no such user exists on the WSL2 host (lap2 correctly uses root); fixed by setting ssh_user='root' to match. bms-4’s p24-dispatcher pubkey was already present in /root/.ssh/authorized_keys on the guest. Verified from bms-4: ssh -i /root/.ssh/p24-dispatcher -o ProxyCommand="cloudflared access ssh --hostname %h" root@windows-dev-ssh.zintegrowana.online succeeds, no OTP prompt, no publickey failure. current_workers in dev_r_server_capacity has been observed stale/mismatched against actual WSL docker-container count — verify via wsl -d Ubuntu --exec docker ps rather than trusting the DB column alone until that’s root-caused.

Environment ID: windows-dev Type: local-dev (worker host) Status: LIVE and dispatchable end-to-end (#6110 / ADR-005 §5b, ADR-006 Phase 1/2 complete) — dev_r_server_capacity.enabled=true, ssh_user=root, scaled for idle-resource utilization.

This is the design of record for a Docker/WSL2 worker on the developer workstation, per the owner decision reconciled in ADR-005 (the Pane detour in #6132 was reverted). It supersedes wsl1.md’s self-polling / native-process architecture (that doc’s hardware spec is reused below; its design is not).

Purpose

The dev workstation’s WSL2 Ubuntu instance, run as the 3rd/4th worker host (after bms-4 and the lap1 laptop) using Docker ephemeral workers — one docker run --rm-style container per job. Unlike lap1’s “idle-window only” framing, windows-dev workers run concurrently with interactive use (this Claude Code session, IDE, browser) — so every container is hard-capped on RAM and CPU to protect the developer’s foreground work (ADR-005 §5b).

It is a variable-uptime overflow node, never load-bearing: it must not hold the dispatcher lease and must not be a dependency for any scheduled job. If the machine is off or the tunnel is down, its unclaimed queue rows simply stay queued for bms-4 (pull-based dispatch on liveness).

Hardware

Real spec (reused from wsl1.md; same physical host, WSL2 allocation):

FieldValue
Labelwindows-dev
Host OSWindows 10 (dev-workstation, see dev-workstation.md)
Guest OSUbuntu 26.04 LTS (WSL2, systemd enabled)
CPU6 cores
RAM46 GB (WSL2 allocation — raised from 31 GB via ~/.wslconfig, weight-tier update 2026-08-11)
Disk954 GB free
InboundCF Tunnel: windows-dev-ssh.zintegrowana.online (no public IP; ADR-005 §3)

Capacity (dev_r_server_capacity — ships DISABLED)

Set by migration supabase/migrations/20260811052102_windows_dev_docker_worker_capacity_6110.sql, superseded by 20260811133644_windows_dev_weight_tiers_update.sql (owner weight-tier redefinition + WSL2 ceiling raised to 46 GB):

FieldValueWhy
enabledtruelive (2026-10-14) — see status banner above
max_workers_prime / max_workers_night4 / 8raised 2026-10-14: the workstation is on 24/7 but rarely interactively used, so night capacity now exceeds day capacity (inverse of the original MVP default)
max_weight_prime / max_weight_nightheavy / heavyraised 2026-10-14 (night was light-only before — backwards, since night is when the host is most idle). playwright is NOT yet enabled — same gating (#7718-style smoke test) as lap2
weight_ram_gbsuperlight=1, light=2, normal=4, heavy=7 (playwright=7, subagent=1, orchestrator=12)owner-defined tiers (2026-08-11); each == the Docker -m hard cap for that weight
reserved_ram_gb2 (of 46)headroom for the interactive session on the same box
emergency_max_workers10raised 2026-10-14 (was 1) — hard cap, not the steady-state cap
claude_accountsclaude-runner, claude-runner-2existing accounts only, no new seats (§4). radieu deliberately excluded (2026-09-01, owner decision) — it’s the developer’s own daily-driver account, and automated dispatch on it would compete with interactive use on this same workstation right when it’s being used

Usable budget: (46 − 2) × 0.80 ≈ 35.2 GB → a light worker uses 2 GB, well under cap; max_weight stays light-only regardless (capacity raised, dispatch ceiling unchanged).

Dispatch path (CF Tunnel inbound — ADR-005 §3)

bms-4 dispatcher (queue-dispatcher-loop.py)
  → CF Tunnel (windows-dev-ssh.zintegrowana.online)
  → scripts/run-windows-dev-worker.sh  ISSUE JOB_TYPE WEIGHT ...
      → docker run --memory <G> --memory-swap <G> --cpus <n>  p24-windows-dev-worker  <args>
          → entrypoint.sh → spawn-worker.sh  (WORKER_ISOLATION=docker, foreground, no systemd)
      → translate outcome → dispatcher's five-code contract

Unlike Linux hosts (dispatcher SSHes spawn-worker.sh directly), windows-dev is reached over its CF Tunnel and the far end runs run-windows-dev-worker.sh. The dispatcher-loop change to invoke this path is implemented (#6163) — queue-dispatcher-loop.py’s spawn_worker() detects a windows-dev/docker-run host, wraps SSH in a cloudflared access ssh ProxyCommand (the lap1 pattern), and launches run-windows-dev-worker.sh instead of spawn-worker.sh. The row still stays enabled=false until the on-host provisioning + soak complete (see the go-live playbook).

Detached, fire-and-forget launch. Because the container runs the Claude worker in the foreground for the whole job (spawn-worker.sh WORKER_ISOLATION=docker), the dispatcher launches it DETACHED (setsid … &) and returns as soon as it is spawned — exactly like the bare-host systemd-run --scope & detach — rather than blocking the 2-min dispatch loop for the job’s lifetime. Completion is self-reported by the in-container queue-row status/heartbeat PATCH (+ the dispatcher’s stale-worker reset as the safety net). The wrapper’s exit-code translation below therefore applies when it is run synchronously (the manual smoke test), not on the live detached dispatch path; synchronous exit-5 account rotation is consequently not available on this path (acceptable for the light-only, single-worker MVP — ADR-005 §5b).

Exit-code / RAM-gate translation contract (the core of #6110 item 2)

Docker signals an OOM-kill with its own code (137 = 128+SIGKILL) and a .State.OOMKilled flag, neither of which the dispatcher’s OOM path (exit 2) recognises. run-windows-dev-worker.sh translates so the dispatcher’s retry/requeue/escalate logic is unchanged:

Container outcomeWrapper emitsDispatcher behaviour
clean run0done
.State.OOMKilled==true OR exit 1372OOM → re-dispatch heavier / small-host reroute
in-container spawn-worker.sh RAM pre-check3 (passthrough)requeue unchanged
needs heavier weight4 (passthrough)escalate tier
subscription blocked5 (passthrough)try next account
git clone/auth failure6 (passthrough)reroute to bms-4
container could not run (docker 125/126/127)255treat as connectivity failure (reroute/backoff)

The -m/--cpus caps (light → 2 GB / 1 CPU; normal → 4 GB / 2 CPUs; heavy → 7 GB / 3 CPUs) are applied by the wrapper and must equal the weight_ram_gb budget above. Tested in scripts/tests/test_run_windows_dev_worker.py; the dispatcher-side path is tested in scripts/tests/test_spawn_windows_dev_dispatch.py.

OAuth token refresh + sub-usage crons (WSL2 guest, added 2026-10-14)

Mirrors lap2’s #7965 pattern (see lap2.md) — the standard Ansible-managed claude-token-refresh cron targets /home/<user>/.claude/ and does not apply to a host whose Docker workers mount credentials from /root/.claude-configs/<account>/ on the WSL2 guest instead. Installed directly on this host (was previously missing — a gap found during the 2026-10-14 go-live check):

  • /etc/cron.d/claude-token-refresh — every 15 min, both claude-runner and claude-runner-2, --uid 1001 (the claude user’s uid inside p24-windows-dev-worker, confirmed via docker run --rm --entrypoint id p24-windows-dev-worker -u claude).
  • /etc/cron.d/claude-sub-usage — every 10 min, both accounts, CLAUDE_SUB_MACHINE=windows-dev so the ClaudeRunnerTokenExpired/ClaudeRunnerTokenExpiringSoon Prometheus gauges label correctly.

Verify: tail /var/log/claude-token-refresh.log / tail /var/log/claude-sub-usage.log inside WSL2 after one cycle.

Real auth probe cron (WSL2 guest, #7965)

claude-auth-synthetic-check.py now supports --config-base <dir> and --machine <label> (#7965, mirroring lap2 — see lap2.md §Real auth probe). Add /etc/cron.d/claude-auth-synthetic-check (create if absent) on the WSL2 guest:

SHELL=/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

*/15 * * * * root python3 /opt/p24-infra/scripts/claude-auth-synthetic-check.py \
  --config-base /root/.claude-configs \
  --machine windows-dev \
  >> /var/log/claude-auth-synthetic.log 2>&1

This iterates all account subdirs under /root/.claude-configs/ (claude-runner, claude-runner-2) and pushes p4_claude_auth_valid{machine="windows-dev",account="…"} to the vps-i1 Pushgateway. Without it, a dead token on windows-dev is caught only reactively (after a job fails with spawn_oauth_expired) rather than by the proactive ClaudeAuthSyntheticFailing alert (30 min).

Verify after one cycle (≤15 min):

# On the WSL2 guest
tail -20 /var/log/claude-auth-synthetic.log   # "Auth VALID for radieu/ecotrans on windows-dev"
# On vps-i1 (Pushgateway)
ssh root@217.154.82.162 "curl -s http://localhost:9091/metrics | grep p4_claude_auth_valid | grep windows-dev"

Secrets access (ADR-005 §8 — per-role, at the container-mount level)

The container inherits the same CLAUDE.md Role Enforcement matrix as a bms-4 worker, enforced at the mount level, not just in-process:

  • secret-manager-role containers only get the SOPS age key mounted in.
  • sys-admin / sys-security get read-only credential access for verification (not yet wired on the docker-run path — a follow-up to #7104).
  • dev-coder and every other role get no SOPS/age-key access at all, even transiently — the key is simply never mounted, so there is nothing to withhold after the fact.

Every research/investigation subagent still carries the mandatory #5544 “No SOPS decryption” clause, identically inside a container.

How it is enforced (#7104). scripts/run-windows-dev-worker.sh reads the ROLE positional arg ($7) and adds a read-only age-key mount only when it is secret-manager: -v $WORKER_AGE_KEY_FILE:/home/claude/.age/p24-infra-keys.txt:ro (the container path spawn-worker.sh exports as SOPS_AGE_KEY_FILE). Every other role builds an empty mount array, so the key is never bind-mounted.

  • WORKER_AGE_KEY_FILE is a host env var set at provisioning — this host’s own automation age-key path, deliberately not hardcoded and distinct from the interactive developer’s personal key. Set it before dispatching any secret-manager job here.
  • A secret-manager job dispatched with WORKER_AGE_KEY_FILE unset or pointing at a missing file fails closed (the wrapper exits 1 — the validation-failure code — rather than launching a SOPS-blind container).

Graph View

Backlinks