AI Risk Management & Data Governance — Ecotrans / p24-infra

Document type: Internal formal record — EU AI Act Art. 9 (risk management) & Art. 10 (data governance) good practice
Scope: All AI systems deployed by the p24-infra project
Last reviewed: 2026-06-16
Next review due: 2027-06-16
Owner: Radek Konarski (p24-infra admin)
Risk classification: All current systems are minimal risk under the EU AI Act — see eu-ai-act-compliance.md

Note on applicability: Art. 9 and Art. 10 are mandatory only for high-risk AI systems (Annex III). This document is maintained as good practice for minimal-risk systems and as a readiness baseline in case any system is reclassified. It also fulfils the has_risk_management and has_data_governance compliance fields in dev_r_ai_systems.

Part A — Risk Management

A.1 Systems covered

System	Service	Risk tier	Model
`workbook-designer`	`audit-engine`	minimal	claude-sonnet-4-6
`report-generator`	`audit-engine`	minimal	claude-sonnet-4-6

A.2 Risk identification & assessment

R-01 — Incorrect workbook design leading to misleading reports

Attribute	Detail
Description	Claude designs a WorkbookSpec with wrong connector params, wrong data sources, or misframed analysis goals. The resulting report misrepresents infrastructure health.
Trigger conditions	Ambiguous action description; poor prompt context; model regression
Likelihood	Low — action descriptions are templated and reviewed at design time; 3 retries with JSON-schema validation
Impact	Low — reports are informational; no automated action is taken; Prometheus and Grafana serve as independent ground truth
Mitigation	(1) WorkbookSpec JSON-schema validation before activation (2) Human operator reviews weekly/daily reports (3) `/reoptimize` endpoint to manually trigger redesign (4) Prometheus alerts fire independently of report output
Residual risk	Negligible

R-02 — AI hallucination in generated report

Attribute	Detail
Description	Claude invents metric values, fabricates alert states, or draws incorrect conclusions from real connector data.
Trigger conditions	Model hallucination; prompt injection in connector data
Likelihood	Low-medium — connector data is structured (JSON/PromQL output); hallucination risk exists on synthesis step
Impact	Low-medium — operator might miss a genuine issue or act on a phantom one
Mitigation	(1) Reports labelled “AI-generated — verify against Grafana before acting” (2) All source data retained in audit.runs output for cross-check (3) Critical alerts come from Alertmanager, not from AI reports
Residual risk	Low

R-03 — Sensitive infrastructure data exposure via AI calls

Attribute	Detail
Description	Infrastructure metrics, service names, or credentials sent to the AI model are intercepted or retained by the AI provider.
Trigger conditions	Network interception; AI provider data retention; accidental secret injection
Likelihood	Low — calls route through claude-proxy on vps-h1 over private network; TLS on all external hops
Impact	Medium — infrastructure topology details could aid a targeted attack
Mitigation	(1) claude-proxy on private vps-h1, authenticated with `CLAUDE_PROXY_SECRET` (2) Claude Max subscription — Anthropic does not train on Claude Max data by default (3) Secret scrubbing: connector params must never include raw credentials (validated in WorkbookSpec schema) (4) Anthropic’s EU data processing and privacy policies apply
Residual risk	Low

R-04 — Token budget overrun causing service cost spike

Attribute	Detail
Description	A runaway ai_workbook action consumes excessive tokens, exceeding monthly budget.
Trigger conditions	Very large connector data payload; retry loop bug
Likelihood	Low — guardrails.py enforces per-action cap (2k–12k tokens) and monthly rolling cap (100k tokens for p24-infra)
Impact	Low — financial (Claude Max flat-rate subscription, no marginal cost); service disruption if monthly cap hit before month end
Mitigation	`guardrails.py` blocks execution when cap exceeded; `audit.increment_monthly_tokens()` RPC tracks usage atomically
Residual risk	Negligible

R-05 — AI system unavailability (claude-proxy or Anthropic API down)

Attribute	Detail
Description	Reports are not generated during an outage.
Trigger conditions	claude-proxy crash; Anthropic service outage; network partition
Likelihood	Low-medium — Anthropic has high availability; claude-proxy is a single-process systemd service
Impact	Low — missed report; Prometheus/Grafana continue operating independently
Mitigation	(1) `ai.py` falls back from claude-proxy to direct `ANTHROPIC_API_KEY` if proxy returns 5xx (2) audit.runs records `status='error'`; retry happens on next scheduled run (3) Audit engine itself has a Prometheus health endpoint (`/metrics`)
Residual risk	Negligible

R-06 — Prompt injection via connector data

Attribute	Detail
Description	Malicious content in a monitored system (e.g. a GitHub issue title, a Prometheus label) attempts to override the AI’s instructions.
Trigger conditions	Adversarial content in monitored data sources
Likelihood	Very low — data sources are internal (Prometheus, Supabase) or authenticated external APIs
Impact	Low — could cause incorrect report output; no code execution or external network calls from AI output
Mitigation	(1) Connector data is structured (JSON/metrics), reducing free-text attack surface (2) AI output is text-only; no tool calls or shell commands are executed from report output (3) WorkbookSpec validation prevents injecting new connector steps via report content
Residual risk	Negligible

A.3 Overall residual risk verdict

Negligible to Low. All identified risks are mitigated to acceptable levels through existing architectural controls. The systems are appropriate for their minimal-risk classification. No additional risk controls are required before August 2026 for these systems.

A.4 High-risk classification triggers

The following scenarios would require re-assessment and full Art. 9 compliance:

Introducing AI that scores, ranks, or evaluates driver performance → Annex III.4(a)
Introducing AI that automatically assigns tasks or routes to human workers → Annex III.4(a)/(b)
Introducing AI that makes creditworthiness assessments about customers → Annex III.5(b)
Introducing AI that manages safety-critical vehicle systems or emergency dispatch → Annex III.2(a)

When any of the above is planned: pause deployment, complete full Art. 9 risk management system documentation, and apply conformity assessment procedures.

A.5 Review schedule

Event	Action
Annual (next: 2027-06-16)	Re-assess all risks; update `last_reviewed` in `dev_r_ai_systems`
New AI system deployment	Complete risk assessment before go-live; add row to `dev_r_ai_systems`
New model version (major)	Re-evaluate R-02 and R-03
Security incident	Immediate re-assessment of R-03 and R-06

Part B — Data Governance

B.1 Data inventory

Data category	Source	Contains PII?	Sent to AI?
Prometheus metrics	vps-i1 node_exporter, cAdvisor	No	Yes (aggregated time series)
Prometheus alert states	Alertmanager	No	Yes (alert names, labels)
GitHub Actions data	GitHub API	No (only run counts, workflow names)	Yes
Wasabi backup status	Wasabi S3 bucket scan	No	Yes (JSON backup manifest)
Supabase queue depths	Supabase `pg_stat_statements`, queue tables	No (counts/depths only)	Yes
Supabase fleet data	`fleet_positions`, `incidents`	Potentially (vehicle IDs)	Yes — see B.3
n8n workflow metrics	Prometheus `n8n_*`	No	Yes
audit.runs output	Supabase `audit.runs`	No	No (stored, not fed back)

B.2 Personal data assessment

Current connectors in production do not process personal data at the point of AI input:

Prometheus metrics: numeric time series, no person identifiers
GitHub API: workflow run counts, commit SHAs — no PII
Wasabi: file names, sizes, timestamps — no PII
Supabase queue depths: integer counts — no PII

Fleet data (Supabase connector — Weekly Fleet Inspection Report):
This connector queries fleet_positions and incidents. These tables may contain:

Vehicle registration plates (indirectly identifies a driver in some jurisdictions)
GPS coordinates linked to a vehicle

Handling: The AI workbook spec for fleet reports must aggregate and anonymise before passing to AI (e.g. counts, status distributions, not raw plate + location tuples). The workbook designer is instructed to use aggregate queries via the Supabase connector row_limit parameter. Direct raw GPS + plate data must never appear in AI workbook input.

This constraint is noted in audit.projects.config ("pii_handling": "aggregate_only") and enforced by workbook design review.

B.3 Data flow diagram

[Prometheus]──────────────────────────────────────┐
[GitHub API]──────────────────────────────────────┤
[Wasabi S3]──────→ audit-engine connectors         │
[Supabase DB]─────  (on vps-h1)                    │
                         │                         │
                         ▼ structured JSON          │
                   claude-proxy:9999                │
                   (vps-h1, local)                  │
                         │ HTTPS                   │
                         ▼                         │
                  Anthropic Claude API              │
                         │                         │
                         ▼                         │
                   AI analysis text                 │
                         │                         │
                         ▼                         │
              audit.runs (Supabase) ◄──────────────┘
                         │
                         ▼
              PDF via pdf-service → email / Google Drive

All data leaves the VPS network only toward:

Anthropic Claude API (TLS 1.3) — EU data processing terms apply
Supabase (PostgreSQL, TLS) — EU region (eu-central-1)
Google Drive (TLS) — for report archival

B.4 Data quality & bias

Training data: We do not train models — we use Anthropic Claude via API/OAuth. Bias in the underlying model is Anthropic’s responsibility (provider obligations under EU AI Act Art. 55).

Input data quality: Prometheus metrics are collected at 15s intervals by node_exporter/cAdvisor with no preprocessing. Data quality depends on exporter correctness. Known gaps:

bms-2 and bms-3 do not yet have node_exporter deployed → those servers are not reflected in AI reports
Thanos query covers both live Prometheus + S3 historical blocks for trend analysis

Connector row_limit: All workbook steps set row_limit to prevent unbounded data payloads.

B.5 Data retention

Store	Data	Current retention	Target
`audit.runs`	All run outputs including connector data	Indefinite	90 days (add cleanup job)
`audit.workbooks`	Workbook specs	Indefinite (versioned)	Keep 3 active versions per action
Anthropic API	Request/response during call	Per Anthropic privacy policy (no retention for Max)	N/A
Google Drive	PDF reports	Indefinite	Annual archival review
Email	PDF attachments	Per email provider policy	N/A

Action item: Create a pg_cron job on Supabase to delete audit.runs rows older than 90 days.

B.6 Access controls

System	Who can access	How
audit-engine	Service (vps-h1)	`SUPABASE_SERVICE_KEY` (env var on vps-h1)
audit.runs	grafana_readonly (read), service_role (write)	Supabase RLS policies
dev_r_ai_systems	grafana_readonly (read), service_role (write)	Supabase RLS policies
claude-proxy	audit-engine only	`CLAUDE_PROXY_SECRET` header (env var)
Anthropic API	claude-proxy only	Claude Max OAuth credentials (`~/.claude/.credentials.json`)
PDF reports (email)	Recipients: `radieu@gmail.com`, `ecotrans.automation@gmail.com`	Email auth
PDF reports (Drive)	Google SA with Drive scope	Per-project Vault secret

No external user has access to AI system inputs, outputs, or intermediate data.

B.7 Third-party processors

Processor	Role	Legal basis	EU adequacy
Anthropic	Processes infrastructure metrics + fleet aggregates to generate reports	Legitimate interest (operational reporting); Claude Max subscription terms	Anthropic’s EU SCCs / DPA applies; Claude Max: no training on inputs
Supabase	Stores AI outputs in `audit.runs`	Contract (Supabase Pro subscription)	EU region (`eu-central-1`); Supabase DPA available
Google (Drive/SMTP)	Report archival and delivery	Contract	EU adequacy decision applies

B.8 Data governance review

Any change to the set of connectors or data sources used by audit-engine workbooks must:

Be assessed against B.1–B.3 above
Confirm no PII is passed raw to the AI model
Update this document

Next full review: 2027-06-16

p24-infra Docs

Explorer

ai-risk-data-governance