p24-infra Docs

gh-runner-queue-registration

3 min read

GH Actions self-hosted runner — worker-queue registration

Issue: #1923 Status: active (p24-infra side); the consuming workflow lives in radieu/whatsup-android-chat-puller

Why

Self-hosted GH Actions runners (e.g. bms4-android) share a physical host (bms-4) with the Claude worker fleet. A heavy CI job — a full Android build, connectedAndroidTest (Espresso/e2e), or a coverage report — saturates the host’s RAM/CPU. If the queue dispatcher spawns Claude workers on that same host while the build runs, both starve.

The fix: a heavy runner job registers itself in Supabase gh_runner_sessions when it starts and deregisters when it finishes. queue-dispatcher-loop.py reads that table each cycle and defers Claude worker dispatch on any server with an active heavy job.

Light/fast jobs (merge commits, changelog updates, label/milestone sync, anything <~30s) do NOT register — the Supabase round-trip overhead is not worth it and would pollute the busy signal.

What counts as “heavy”

Heavy (register)Light (no register)
assembleRelease / assembleDebug full buildPR merge commit
connectedAndroidTest (e2e, Espresso)Changelog update
test with coverage (jacocoTestReport)Label/milestone sync
Any job >~2 min estimatedAny job <~30s estimated

Components

ComponentLocationRole
gh_runner_sessions tablesupabase/migrations/20260628110358_gh_runner_sessions.sqlOne row per heavy runner job (running → done/failed)
runner-register.shscripts/runner-register.shCalled by CI steps: start / finish [done|failed]
_runner_busy() gatescripts/queue-dispatcher-loop.pyPer-server check; skips Claude dispatch while a heavy job runs
build-apk.yml stepsradieu/whatsup-android-chat-puller (separate repo)Register/deregister around the build

How it works

  1. Build starts → first workflow step runs runner-register.sh start, which POSTs a row to gh_runner_sessions (status='running') and stashes the row id in /tmp/runner-session-id.
  2. Dispatcher cycle → for each server, _runner_busy(server_label) queries gh_runner_sessions?server_node=eq.<label>&status=eq.running&started_at=gte.<now-2h>. If any row is returned, the dispatcher logs deferring Claude worker dispatch this cycle and continues to the next server — no job claim is consumed, queued jobs stay queued for the next cycle / another server.
  3. Build finishes (always, even on failure via if: always()) → runner-register.sh finish <status> PATCHes the row to status='done'|'failed' with finished_at.

Stale-row self-heal (2h cutoff)

A runner that crashes mid-build never calls finish, leaving a stale running row that would block dispatch forever. Both the read path (_runner_busy, RUNNER_STALE_MINUTES = 120) and the migration treat rows older than 2 hours as stale and ignore them. Manual cleanup if needed:

UPDATE gh_runner_sessions SET status='failed', finished_at=now()
  WHERE status='running' AND started_at < now() - interval '2 hours';

Keep RUNNER_STALE_MINUTES (dispatcher) and the 2h interval (migration comment) in sync if you change one.

Fail-soft guarantees

  • runner-register.sh always exits 0 — a Supabase hiccup must never fail a build. A hard curl failure emits a best-effort Discord warning (P24_DISCORD_INFRA_SCRIPTS_ERRORS_WEBHOOK_URL) but does not open a GH issue (transient telemetry blips would be alert noise).
  • _runner_busy() returns [] (not busy) on any error — a telemetry problem never wedges dispatch.

Secrets

runner-register.sh reads SUPABASE_URL and SUPABASE_SERVICE_ROLE_KEY from the environment only (never printed). In CI they come from GH Secrets in the consuming repo. Values originate from secrets/monitoring.env.sops (SUPABASE_URL) and secrets/n8n-bms4.env.sops (SUPABASE_SERVICE_ROLE_KEY) — SOPS+age, never committed in plaintext.

Consuming-repo wiring (separate task)

The radieu/whatsup-android-chat-puller workflow change and its GH Secrets are out of scope for the p24-infra repo and tracked as a follow-up. The build job must:

  1. Add SUPABASE_URL and SUPABASE_SERVICE_ROLE_KEY to the repo’s GH Secrets.
  2. Add a first step run: /opt/p24-infra/scripts/runner-register.sh start.
  3. Add a final step with if: always(): run: /opt/p24-infra/scripts/runner-register.sh finish ${{ job.status == 'success' && 'done' || 'failed' }}.

See issue #1923 for the full build-apk.yml snippet.

Graph View

Backlinks