infrastructure-standard

p24-infra Infrastructure Standard

Every service or credential added to p24-infrastructure must be registered in Supabase dev_r_services with the appropriate element_type. Compliance is tracked in that table and visualised on the p24-infra Health Dashboard in Grafana.

---

Requirements — Services

The following six requirements apply to every service (containers, SaaS, servers). Credentials use only requirements 3 and 6 — see Credentials below.

1. Workbook

A docs/{service-name}-operations.md file covering:

• Architecture (containers, ports, volumes, networks)
• Config management (what lives in repo vs. on server, how secrets are injected)
• Deployment procedure (fresh install + update)
• Upgrade procedure

2. Monitoring

At minimum:

• Prometheus alert: {Service}Down — critical, fires within 2 min of container disappearing
• Prometheus alert: {Service}HighRestarts — warning, >2 restarts/hour
• Blackbox HTTP probe on the public HTTPS URL (if service has one)

3. Backup

• Automated backup on a schedule (daily at minimum)
• Backup stored off-server (Wasabi S3, bucket p24-infra, folder {service-name}/)
• Retention policy defined and enforced

4. Restore

• Restore procedure documented and tested in the workbook
• Restore from Wasabi must be achievable in under 30 minutes

5. Healthcheck

• Docker healthcheck: defined in the compose service, OR
• External HTTP probe via blackbox-exporter, OR
• Both

6. Password Rotation

• All service passwords / API keys registered in dev_r_services as element_type='credential'
• Rotation procedure documented in workbook
• Rotation frequency defined per the table in the Credentials section below

---

Compliance Levels

Value	Meaning
yes	Requirement fully met
partial	Partially implemented — see notes
no	Not implemented
n/a	Not applicable for this element type (e.g. credentials have no workbook)

---

Credentials as Infrastructure Resources

Every credential — API key, password, token, OAuth secret, SSH key, webhook URL — is a first-class infrastructure resource. All credentials must be registered in dev_r_services with element_type = 'credential' and service_type = 'secret'.

Required fields for credentials

Field	Value
service_name	Uppercase env-var key name (e.g. GRAFANA_ADMIN_PASSWORD)
element_type	'credential'
service_type	'secret'
criticality	critical / high / medium / low
description	What the credential grants access to
rotation_freq	'90d' / '180d' / '365d' / NULL (no rotation needed)
last_rotated	Date of last rotation, or NULL if never formally rotated
next_due	last_rotated + rotation_freq, or NULL
compliance_password_rotation	'yes' if current, 'no' if overdue/bootstrap, 'n/a' if no rotation needed
compliance_backup	'yes' if backup storage exists, 'no' if single location only
compliance_workbook	'n/a'
compliance_monitoring	'n/a'
compliance_restore	'n/a'
compliance_healthcheck	'n/a'
compliance_notes	Primary + backup storage locations

Rotation frequency recommendations

Criticality	Recommended freq
critical	90 d
high	90–180 d
medium	180 d
low	365 d
webhooks / public keys	— (no rotation)

When adding a new credential

1. INSERT a row into dev_r_services with the fields above
2. Open a bootstrap rotation-log entry: python3 scripts/rotation-log-entry.py open --secret KEY_NAME --reason "bootstrap" (writes to dev_r_rotation_log)

When rotating a credential

1. Rotate the value in all storage locations (VPS .env, GH Secret, .env.local)
2. Update the registry:

   UPDATE dev_r_services
   SET compliance_password_rotation = 'yes',
       last_rotated = CURRENT_DATE,
       next_due     = CURRENT_DATE + INTERVAL '<rotation_freq>',
       updated_at   = now()
   WHERE service_name = '<KEY_NAME>'
     AND project_id   = 'p24-infra';

3. Close the rotation-log entry: python3 scripts/rotation-log-entry.py close "$LOCK_ID" (writes to dev_r_rotation_log)

---

Adding a New Service — Checklist

When adding a new service to p24-infra:

1. INSERT a row in Supabase dev_r_services (project_id=‘p24-infra’) with compliance fields set to no
2. Create services/{name}/docker-compose.yml + .env.example
3. Implement requirements in order: monitoring → healthcheck → backup → workbook → restore test → password rotation
4. Update dev_r_services row as each requirement is met
5. Verify the p24-infra Health Dashboard reflects the changes

---

Published container ports (#7660)

Docker publishes container ports through its own DNAT rules, evaluated before ufw — so a port published on 0.0.0.0 is internet-reachable even with ufw active and no allow rule. ufw only guards host-process listeners, never container-published ones.

Rules for every Docker-published port:

1. Default to 127.0.0.1: in the compose ports: mapping. If a service only needs to be reached by other containers, drop the host publish entirely and use the Docker network.
2. WAN exposure (0.0.0.0/::) requires justification. Any WAN-bound port must have an entry in the host allowlist (<host>/docker-port-allowlist.conf) recording its consumer and owner, and must be one of:
   • public — open to the internet by design (e.g. Traefik 80/443),
   • fleet — WAN-bound but managed by the DOCKER-USER firewall (scripts/docker-user-firewall.sh) via a per-port source allow list (PORT_ALLOW_<port>). Used for exporters scraped cross-fleet by vps-i1 Prometheus/blackbox, which cannot be 127.0.0.1-bound without blacking out monitoring, or
   • unrestricted — WAN-bound and deliberately NOT firewall-managed because a legitimate client connects from outside the fleet (e.g. RabbitMQ 5672 ← bms-1 production AMQP). Must be justified in the allowlist with its external consumer.
3. Observe before dropping. The firewall’s default (install) is count-only — it counts connections per source but drops nothing. Enforcement (DROP) is a separate explicit step taken only after the counters confirm every real source, and never on an unrestricted port.
4. Regression guard: scripts/check-docker-port-exposure.sh fails when a container publishes a WAN port not in the allowlist. Run it from CI / the invariants audit.
5. Mechanism + observe/enforce/verify/rollback: docs/playbooks/docker-port-firewall.md.

Current Compliance

See live status: grafana.vps-i1.infra.zintegrowana.online → p24-infra Health dashboard.