p24-infra Docs

5836-compliance-audit-dispatcher-route

12 min read

Design — Compliance-audit issues need a dispatcher route (#5836)

Status: Design (plan-first) · Author: dev-coder worker · Date: 2026-08-07 Issue: #5836 Related: #1972 (initial compliance run) · docs/playbooks/compliance-audit-policy.md · scripts/compliance-audit-due-check.py · .github/workflows/compliance-audit-due-check.yml

1. Problem

[Audit Due] compliance issues pile up unexecuted — nothing in the automation picks them up. Discovered during /issues-queue-all triage on 2026-08-07 (6 open: #5833, #4544, #5085, #5086, #4502, #4543). The root cause is two independent gaps, not one:

Gap A — no enqueue path (the issues never reach the queue)

compliance-audit-due-check.py creates each [Audit Due] issue with labels=["compliance-due", <audit_type>] and no milestone, using GH_TOKEN = secrets.GITHUB_TOKEN — so the issue’s author is github-actions[bot].

The only path that turns an issue into a dev_r_worker_queue row is .github/workflows/dispatch-to-queue.yml, which fires on issues.{opened,milestoned,reopened}. That workflow has an explicit bot-author gate:

if [[ "$ACTOR" == ... || "$ACTOR" == "github-actions[bot]" ]]; then
  ... else echo "Bot-authored ${EVENT_NAME}.${ACTION} event by $ACTOR -- skipping"; exit 0

A bot-authored issues.opened is skipped, so the audit-due issue is never auto-milestoned to Triage, the milestoned event never fires, and it is never queued. This is why they accumulate silently past their due dates.

Gap B — no executor (wrong job shape even if enqueued)

Even if an audit-due issue were queued, the only job types spawn-worker.sh accepts are:

dev-issue|continue-issue|infra-alert|infra-task|review-pr|review-plan|nc-alert-batch|alert-triage-batch|secret-manager
  • dev-issue → worker-issue.md: designs + implements code + opens a PR. An audit produces a DB row + issue comments, not a code change — wrong shape.
  • infra-task → infra-task-request-worker.md: built for discrete SSH/ops commands, not for “read a policy playbook, run a domain’s heterogeneous checks across Supabase + psql + mongosh + SSH + Grafana, interpret the results, and write a narrative audit row”.

There is no compliance-audit job type and no worker prompt that knows the audit protocol.

This design (#5836) is deliberately plan-first: it introduces a new pipeline component. The implementation is split into follow-up issues in §7 (some are dev-coder, some are infra-task/deploy and out of a dev-coder session’s scope).

2. Background — what “running a domain’s checks” involves

docs/playbooks/compliance-audit-policy.md defines 12 domains. Each domain section lists concrete checks with SQL/CLI to run and a result of pass | fail | partial | deferred | in_progress, plus a mandatory dev_r_compliance_audits row and a [Compliance Gap] issue per FAIL/PARTIAL (policy §Audit Execution Process, Steps 1–5).

Crucially, the check surfaces are heterogeneous and mostly server-side:

DomainRepresentative checkAccess needed
D4 SecuritySSH authorized_keys on 6 servers; CF token scopes; Supabase RLSSSH (all servers), CF API, psql
D10 DB Performancepg_stat_statements on Supabase PG; MongoDB system.profile on bms-2 PRIMARYpsql on bms-4, mongosh on bms-2 (SOPS conn strings)
D12 CapacityRAM/disk headroom; MongoDB growth runway; queue throughputPrometheus/Grafana, psql
D2 InfraOS versions; Docker image age/CVE; UFW allowlist; dev_r_services completenessSSH, psql
D6 EU AI Actdev_r_ai_systems completeness vs docs/eu-ai-act-compliance.mdpsql, repo read
D8 CostMonthly infra cost vs baseline; Claude/Wasabi spendpsql, exporter metrics

Many checks require judgment (“review admin-actions-investigation.md for undocumented changes”, “spot-check 10% of playbooks against live state”, “no deprecated tools referenced in CLAUDE.md”). This is the decisive constraint for §4.

3. “deferred” decoded + staleness triage of the 6 backlog issues (issue point 3)

Queried live dev_r_compliance_audits on 2026-08-07 before proposing any wiring.

What deferred means operationally

The #1972 initial run (2026-06-28) wrote one row per domain but did not actually run the checks for the server-dependent domains — the running session lacked psql/SSH/mongosh or the check was destructive (DR drill). Their summary fields say so verbatim, e.g.:

  • D10 deferred: “Deferred: pg_stat_statements analysis needs psql on bms-4; MongoDB slow op…”
  • D4 deferred: “Deferred: SSH authorized_keys (6 servers), GitHub collaborators, Cloudflare…”
  • D3 deferred: “Deferred: backup-restore test and bms-1 DR drill are destructive operations…”

Each deferred row kept next_audit_due set (2026-07-28, now in the past), so the daily compliance-audit-due-check.py re-fires an [Audit Due] issue for it every day. deferred = “acknowledged as due, but the checks need a capability the session did not have.” It is not a terminal result; it is a promissory note that the audit still owes real execution.

The 6 backlog issues are mostly already superseded

The due-check dedups by the scope string, not by domain/audit_type. A domain re-audited later under a different scope label (e.g. "D4 … 2026-07-28 follow-up" vs "D4 … initial run (#1972)") produces a fresh row with a future next_audit_due — but it does not clear the old #1972-scoped deferred row, whose past due date keeps re-firing. So 5 of the 6 issues point at a stale #1972 row that has de-facto been superseded:

IssueDomainLatest real audit rowVerdict
#5833D4 Securitypartial 2026-08-06, next due 2026-09-05 (via #4545)superseded → close
#4544D2 Infrapartial 2026-07-29, next due 2026-10-27superseded → close
#5085D6 EU AI Actpartial 2026-08-01, next due 2027-02-01superseded → close
#5086D8 Costpartial 2026-08-01, next due 2026-08-31superseded → close
#4543D12 Capacitypartial 2026-07-29, next due 2026-08-28superseded → close
#4502D10 DB Perfdeferred 2026-06-28 is still the latest row for D10genuinely due — needs psql + mongosh

Only #4502 (D10) is real outstanding work. The other five should be closed as superseded and their stale 1972-scoped rows should have next_audit_due cleared to stop the daily re-fire (§6). This staleness/dedup bug is itself a defect to fix before wiring these into any new pipeline — otherwise the new executor would dutifully re-run already-completed audits.

4. Options considered

Option (a) — new job_type='compliance-audit' + dedicated worker prompt ✅ recommended

A new job type dispatched to a bms-4 sys-admin/infra worker running a new infra/agent-prompts/compliance-audit-worker.md, which reads the policy for the domain named in the audit-due issue, runs that domain’s checks, writes the dev_r_compliance_audits row, files [Compliance Gap] issues, and closes the audit-due issue. No PR (audits are not code changes).

Option (b) — scheduled GH Action / cron script per domain, no LLM

Reject as the primary mechanism, for three reasons:

  1. Judgment-heavy checks can’t be scripted cheaply. “Review admin-action log for undocumented changes”, “spot-check playbooks against live state”, “no deprecated tools referenced in CLAUDE.md/agent-prompts” are LLM-shaped. A bespoke script per check would be N brittle scripts that still can’t do the interpretation the policy asks for.
  2. The purely-mechanical checks are already covered elsewhere and are not the ones piling up: backup freshness, overdue rotations, RS health, queue health are Prometheus/alert-driven (policy §Audit Calendar “Continuous / automated”). The backlog is the periodic, cross-system, judgment domains (D2/D4/D6/D8/D10/D12) — exactly Option (a)‘s sweet spot.
  3. Cross-system reach. One audit spans Supabase + psql + mongosh + SSH + Grafana + repo/SOPS inventory. A generic Claude worker with the infra-task capability set already has that reach; scripts would each need their own credential plumbing.

Hybrid note: Option (a) is the executor; the mechanical sub-checks it runs should still call existing scripts/metrics where they exist (don’t re-derive backup freshness in the LLM). Option (a) does not preclude later moving any single fully-mechanical domain to a pure cron — it’s the right default for the heterogeneous backlog today.

5. Recommended design (Option a) — full spec

5.1 Role & affinity — this is a sys-admin/infra worker, NOT dev-coder

Per §2 the checks need psql on bms-4, mongosh on bms-2, SSH to all servers, and SOPS-read for connection strings. That is the infra role on bms-4 capability set — identical to infra-task. Therefore:

  • compliance-audit is a hard-affinity job type (bms-4 only — it is the sole node with production SSH keys). It must be added to the dispatcher’s _HARD_PREF_JOB_TYPES and always dispatched with server_preference='bms-4', exactly like infra-task (see the #3746 note in worker-issue.md Step 0b-ROLE about keyless hosts stranding hard-affinity jobs).
  • SOPS is read-only for this worker (connection strings only) — it never writes secrets/*.env.sops. Any credential finding becomes a [Compliance Gap] issue, not an inline rotation (that stays secret-manager territory).

5.2 New worker prompt — infra/agent-prompts/compliance-audit-worker.md

Model it on infra-task-request-worker.md (same Step 0-ROUTE / session-registration / claim-mutex preamble, same REST-fallback helpers), but replace the “execute the ops command” body with the audit protocol — a thin wrapper over compliance-audit-policy.md §Audit Execution Process:

  1. Parse the audit-due issue: audit_type, scope, last_audit_row_id (from the issue body the due-check writes).
  2. Staleness guard (fixes the §3 class permanently): query the latest dev_r_compliance_audits row for this audit_type/domain. If a newer row already exists with next_audit_due in the future, do not re-run — comment “superseded by row <id> (next due <date>)”, clear the stale row’s next_audit_due, close the audit-due issue. Exit success.
  3. Otherwise read the matching Domain section of compliance-audit-policy.md and run its listed checks (the SQL/CLI is in the policy — psql via SUPABASE_DB_URL from SOPS, mongosh on bms-2, SSH where required). Never print secret values (policy §Step 2 + global rule).
  4. Insert exactly one dev_r_compliance_audits row (result, summary, findings, action_taken, doc_url, next_audit_due per the frequency table).
  5. For each FAIL/PARTIAL finding: create a [Compliance Gap] issue (policy §Step 4) with labels=compliance-gap,<audit_type>, then set gh_issue on the audit row.
  6. Close the audit-due issue; mark the queue row done. No branch, no PR.

Destructive checks (D3 restore test, bms-1 DR drill) are out of scope for the autonomous worker — it records result='deferred' with a summary naming the human/scheduled action and files a human-action issue, mirroring how the #1972 rows already handle them.

5.3 spawn-worker.sh wiring (dev-coder-implementable)

  • Add compliance-audit to the job_type validation case (line ~74).
  • Job-type→persona map (line ~334): compliance-audit) CLAUDE_ROLE="sys-admin" ;;.
  • Prompt selection (both the infra role branch ~430 and the JOB_TYPE fallback ~471): compliance-audit) AGENT_PROMPT_FILE="compliance-audit-worker.md" ;;.
  • Extend scripts/tests/test_worker_role_capabilities.py for the new job_type→role mapping.

5.4 meta-dispatcher classification (dev-coder writes; deploy is infra-task)

infra-src/meta-dispatcher/src/classify.ts §0 — treat it like infra-task:

if (jobType === "infra-task" || jobType === "secret-manager" || jobType === "compliance-audit") {
  return { role: "infra", repo_context: null, server_preference: "bms-4" };
}

Add compliance-audit to the dispatcher’s _HARD_PREF_JOB_TYPES. The CF Worker redeploy (npm run deploy, wrangler auth) is a deploy op → follow-up infra-task issue, not this PR.

5.5 Enqueue path — new compliance-audit-dispatch.yml (fixes Gap A)

Mirror the proven nc-alert-instant-dispatch.yml model (it already dispatches bot-authored issues with no bot-author gate):

  • Trigger: issues: types: [labeled] gated on github.event.label.name == 'compliance-due' and issue open and not human-action; plus a schedule backstop (e.g. */30 * * * *) that sweeps any open compliance-due issue lacking a live queue row.
  • Runner: [self-hosted, bms4]. POST to ${QUEUE_API_URL}/queue-issue with {"issue_number": <n>, "repo": "radieu/p24-infra", "job_type": "compliance-audit", "weight": "light", "role": "infra"} (201 queued / 200 dedup-skip).
  • The per-issue mutex (agent_tasks claim) + meta-dispatcher dedup.ts already prevent double dispatch — no new idempotency logic needed.

Alternative (simpler, one fewer workflow): have compliance-audit-due-check.py POST to /queue-issue right after it creates/finds each due issue (add QUEUE_API_URL/QUEUE_API_KEY to the workflow env). Rejected as primary because it couples the “notice” step to the “dispatch” step and misses issues created before the change; the labeled-dispatch workflow also re-queues on manual re-label. Either is acceptable — the labeled workflow matches the established nc-alert pattern and is the recommendation.

5.6 Compliance / registry

compliance-audit-worker is a new automated AI system → register in dev_r_ai_systems + docs/eu-ai-act-compliance.md, and add a dev_r_services row + this ops doc reference (CLAUDE.md §Compliance). Risk classification: minimal/limited (internal ops automation, human-reviewable output, no Annex III trigger) — confirm during implementation.

6. Remediation of the current backlog (do before/with wiring)

  1. **Close #5833, #4544, #5085, #5086, 4543 as superseded (§3 table), each with a comment linking the newer audit row + its future next_audit_due.
  2. Clear next_audit_due on the five stale 1972-scoped deferred rows so the daily due-check stops re-firing them (an UPDATE on dev_r_compliance_audits, not the priorities tables — allowed).
  3. Re-run D10 (#4502) for real once the worker exists — it is the only genuinely-due domain and is exactly what the new worker is for (psql on bms-4 + mongosh on bms-2).
  4. Fix the dedup-by-scope bug at source: either key the due-check on (audit_type, domain) with a stable scope, or have every re-audit reuse the domain’s canonical scope string so a new run supersedes the old row instead of orphaning it. Tracked as a follow-up (§7).

Steps 1–2 are operational cleanup (issue closes + one UPDATE); this plan PR recommends them but does not execute them, to keep the design PR clean — they belong to the implementation issues.

7. Implementation plan (follow-up issues)

#WorkRoleNotes
1compliance-audit-worker.md prompt + spawn-worker.sh wiring + testdev-coder§5.2, §5.3
2classify.ts + _HARD_PREF_JOB_TYPES change and CF Worker redeploydev-coder (code) → infra-task (deploy)§5.4
3compliance-audit-dispatch.yml workflowdev-coder (needs workflow-scope push — see docs/playbooks/worker-push-workflow-files.md)§5.5
4Backlog cleanup: close 5 superseded issues + clear stale next_audit_dueinfra-task / sys-admin§6.1–6.2
5Fix the due-check dedup-by-scope staleness bugdev-coder§6.4
6Register worker in dev_r_ai_systems + dev_r_services + EU AI Act docdev-coder§5.6

Recommended sequencing: 1 → 2 → 3 (feature online) → 4 (cleanup) → 5 (prevent recurrence) → 6 (compliance). Create via /new-issue per CLAUDE.md; not auto-created here to avoid dispatch churn from a background worker.

8. Regression risks

  • Half-wired feature on main. If spawn-worker.sh/classify.ts accept compliance-audit before the worker prompt + dispatch workflow ship, a stray row would fail. Land §7-1 (worker + spawn wiring) and §7-3 (dispatch) together, or keep the job type undispatched until the prompt exists. This is why the pieces are staged, not merged piecemeal.
  • Hard-affinity stranding. Omitting server_preference='bms-4' lets a keyless host claim the row and strand it (#3746). §5.4 adds it to _HARD_PREF_JOB_TYPES.
  • Double-run / infinite re-audit. Mitigated by the §5.2 staleness guard + existing per-issue mutex; the §6.4 dedup fix removes the source.
  • Secret exposure during checks. Audits touch connection strings and SSH. The worker inherits the absolute never-print-secret rule (policy §Step 2; global CLAUDE.md). Findings reference key names only.
  • Destructive checks auto-run. Explicitly excluded (§5.2) — recorded deferred + human-action.

9. DB schema changes

None. Uses the existing dev_r_compliance_audits, dev_r_worker_queue, agent_tasks, dev_r_ai_systems, dev_r_services tables.

Graph View