p24-infra Docs

pinbox24-w3-tasklogs-retention-cleanup

5 min read

Playbook: Pinbox24 W3 — tasklogs retention cleanup (import-run logs >3 months)

Type: Maintenance / Data Cleanup Service: w3_db.tasklogs (import-run LOG collection) Container: v32-prod @ bms-1 (94.23.26.113) Script: scripts/tasklogs-retention-cleanup.js GH tracking: #4834 (split from #2690)

What this cleanup does

Soft-deletes (archives, reversibly) old import-run log records in w3_db.tasklogs — one doc per import job run (activityName, status, createdAt/updatedAt, registredDataTime). It sets on each matched doc:

archived:   true
archivedAt: <ISODate at execute time>

Match criterion: createdAt < now − N months (default N = 3) AND not already archived: true.

What it deliberately does NOT touch

Not touchedWhy
regRecordsThe actual imported client register data (7.5M docs, live RESO data). Age-deleting there is irreversible production data loss. The architect confirmed on #4834 (2026-08-11) regRecords was never the intended target. The script hard-codes tasklogs and never queries regRecords.
Any S3 / Wasabi objecttasklogs stores no S3 object key — the source Excel is read transiently and S3 storage is the s3-v32-prod-reso sidecar’s job (see reso-import-post-incident-verify.md §S3 leg). There is no per-record “file” to delete when the target is tasklogs.

Why createdAt (not updatedAt / registredDataTime)

  • createdAt — ISODate, $lt-safe, marks when the import run was recorded → truest “age of the import run”.
  • updatedAt — ISODate but moves on later status transitions, so it over-retains a run whose last touch is recent.
  • registredDataTime — a display string "YYYY-MM-DD HH:mm:ss", not $gt-safe (pinned schema, reso-import-post-incident-verify.md).

Docs missing createdAt are never archived (we do not age a record we cannot date); the dry-run reports their count for a separate human decision.

Compliance

No compliance retention requirement is known for tasklogs (repo-wide grep finds retention/stakes language only for regRecords). Re-confirm this is still true before an execute run — if a tasklogs retention/audit obligation has since been introduced, stop and escalate.

When to run

  • On-demand / periodic — when tasklogs growth warrants trimming the import-run log.
  • Not wired into any cron or compose; invoked manually per this playbook.

Prerequisites

  • v32-prod running on bms-1 with w3_db connection env (V32_PMONGODB_URL / PMONGODB_URL / MONGODB_URL).
  • mongoose available in the container (it is — it is the W3 backend’s own dep).
  • Dry-run report reviewed before any --execute (mandatory).

Steps

Every mutating ssh + docker command below must carry a # PLAYBOOK: annotation (.claude/hooks/pre-bash-safety.sh enforces this). This is a sys-admin / infra-task execution step — the dev-coder who authored the script does not run it against production.

Step 1 — Copy the script into the container

# PLAYBOOK: pinbox24-w3-tasklogs-retention-cleanup.md
scp scripts/tasklogs-retention-cleanup.js root@94.23.26.113:/root/tasklogs-retention-cleanup.js
ssh root@94.23.26.113 'docker cp /root/tasklogs-retention-cleanup.js v32-prod:/app/tasklogs-retention-cleanup.js'

Step 2 — Dry run (MANDATORY)

# PLAYBOOK: pinbox24-w3-tasklogs-retention-cleanup.md
ssh root@94.23.26.113 "docker exec v32-prod sh -c 'cd /app && node tasklogs-retention-cleanup.js --dry-run 2>&1'"
ssh root@94.23.26.113 'docker exec v32-prod cat /tmp/TASKLOGS_RETENTION_REPORT.json'

Verify in the report:

  • candidateCount — plausible number of import-run logs older than the window.
  • candidateByActivityName — expected values (Import poprawny, Import cofnięty, Koniec importu / import end, …); nothing surprising.
  • missingCreatedAt — if non-zero, decide separately whether those need handling (the script never touches them).
  • retentionMonths / cutoff — the intended window.

Do not proceed if the numbers look wrong — investigate first.

Step 3 — Execute (only after review)

# PLAYBOOK: pinbox24-w3-tasklogs-retention-cleanup.md
ssh root@94.23.26.113 "docker exec v32-prod sh -c 'cd /app && node tasklogs-retention-cleanup.js --execute --confirmed 2>&1'"

To use a different window, pass --months N on both the dry-run and the execute.

Step 4 — Copy the audit log OUT (before any container restart)

/tmp/ inside the container is ephemeral — a restart wipes it. Copy the manifest out.

# PLAYBOOK: pinbox24-w3-tasklogs-retention-cleanup.md
ssh root@94.23.26.113 "docker cp v32-prod:/tmp/TASKLOGS_RETENTION_AUDIT_LOG.jsonl /root/tasklogs-retention-$(date +%Y%m%d).jsonl"

Each audit line records the _id, activityName, status, createdAt, and archivedAt of an archived doc — the manifest used for rollback.

Step 5 — Verify

# PLAYBOOK: pinbox24-w3-tasklogs-retention-cleanup.md
ssh root@94.23.26.113 "docker exec v32-prod sh -c 'cd /app && node tasklogs-retention-cleanup.js --dry-run 2>&1 | grep Candidates'"

Expected Candidates (createdAt < cutoff): 0 immediately after an execute run (barring docs newly aging past the cutoff).

Rollback / Recovery

Soft-delete is reversible — un-set the flag.

  • Roll back everything the script ever archived: 
    db.tasklogs.updateMany({ archived: true }, { $unset: { archived: "", archivedAt: "" } })
  • Roll back only a specific run (scoped by the audit log’s _id list): 
    db.tasklogs.updateMany({ _id: { $in: [/* ObjectIds from the audit log */] } },
                       { $unset: { archived: "", archivedAt: "" } })

If an actual hard delete of archived logs is ever wanted, that is a separate, deliberate decision — this script intentionally never physically removes docs.

Error handling

  • On connect/query failure the script writes /tmp/TASKLOGS_RETENTION_ERRORS.json and posts a Discord embed via P24_DISCORD_INFRA_SCRIPTS_ERRORS_WEBHOOK_URL (if set in the container env). If the webhook var is unset, the error is only on stderr — check the command output.
  • The --execute write is a single idempotent updateMany; re-running is safe (the candidate query excludes archived: true).
  • Per the repo Error Notification Standard, on any failed run also file a GH issue in radieu/p24-infra labelled bug (the interactive operator does this; the Discord embed is automatic when the webhook var is present in the container env).

Known limitations

  • Old container Node (10.x/12.x) — the script avoids optional chaining / modern syntax (same constraint as scripts/office-retention-cleanup.js).
  • The script reads the Mongo URI from container env vars only — it never prints the URI.

History

DateAction
2026-08-11Script + playbook authored (#4834). Scope confirmed tasklogs (not regRecords), soft-delete default. Not yet executed against production.

Graph View