w3-persistent-patches-committed-2026-07-08

W3 Persistent Patches — Committed to Source Control + CI Deploy Step Added (2026-07-08)

Field	Value
Platform	W3 (w3.pinbox24.com, bms-1) — v32-prod + s3-v32-prod
Trigger	Follow-up from PR #3248 (w4 uploadAwsS3.helper.js fix), which explicitly flagged w3 as having the same category of risk (docs/pinbox24/incident-2026-07-08-w4-signed-url-broken.md §Follow-ups)
Status	RESOLVED — files committed, CI deploy step added. Read-only fetch from bms-1; no production files were modified.
Related	docs/playbooks/pinbox24-s3-wasabi-bms1.md §7 (node-config architecture) + §8 (Open Issues #1, #2, #7), docs/playbooks/pinbox24-w3-operations.md

Problem

docs/playbooks/pinbox24-s3-wasabi-bms1.md and docs/playbooks/pinbox24-w3-operations.md both documented several server-side patch files on bms-1 that override compiled/dist app code in the v32-prod and s3-v32-prod containers — volume-mounted per infra-src/pinbox24/w3/docker-compose.yml (already committed) — but the actual patch file contents were never committed to this repo. They existed only as hand-edited state on bms-1’s disk, with no diff, no review, and no way to redeploy them consistently. secrets-sync.yml’s sync-pinbox24-w3 job also had no step at all that ships a persistent-patches/ directory to bms-1 — only docker-compose.yml itself was deployed.

This is the same failure class fixed for w4 in PR #3248.

Files fetched and reviewed (read-only, credential-safe procedure)

All four files referenced by the volume mounts in infra-src/pinbox24/w3/docker-compose.yml were fetched from bms-1 via scp into a local scratch path, then scanned for credential-shaped strings (accessKeyId, secretAccessKey, password, mongodb://user:pass@..., AWS key prefixes, long base64-like secrets) before being viewed. Zero credential-value hits were found in any of the four files — no redaction was necessary and no code changes were needed to remove hardcoded secrets (unlike w4’s uploadAwsS3.helper.js, which required rewriting three functions to read from env vars).

File	Mounted into	Container path	What it does	Credential handling
persistent-patches/local.js	s3-v32-prod	/app/config/local.js	node-config override — resolves Wasabi S3 config + MongoDB DB_URI	Already 100% process.env.* (s3Bucket_endpoint, s3Bucket_BUCKET_NAME, s3Bucket_api_accessKeyId, s3Bucket_api_secretAccessKey, DB_URI) — no hardcoded values
persistent-patches/controller.js	s3-v32-prod	/app/build/src/api/storage/controller.js	Upload handler; catches Mongoose .save() failures (rs0 migration incompatibility, playbook §8 item 7) and returns 200 with the Wasabi upload result instead of erroring	Uses config.get('upload.*'), which resolves through local.js → env vars. No literals
persistent-patches/filesUpload.js	v32-prod	/app/app-backend/controllers/filesUpload.js	Full controller file; includes the FIX-2026-06-30 mirror-insert (writes file metadata to w3_db via mongojs so excel_import can find it) and the 2026-07-06 regId/recId query-param fallback	db/fileConfig come from require('../config/mongo_db').get() — a shared app config module, not this file. No literals
persistent-patches/crudService.js	v32-prod	/app/app-backend/helperFunction/crudService.js	Large helper module (2026-07-06 Fix B2: callback() on files without a bucket in an async.forEach)	Redis client uses process.env.REDIS_PORT/process.env.REDIS_HOST, and reads process.env.REDIS_PASSWORD only to conditionally call .auth() — no literal password. db again via shared mongo_db config module

None of the four files needed conversion for credentials — they were already environment-variable-driven (or don’t touch credentials directly).

Second finding: hardcoded customer data (not a credential, but not appropriate for git either)

While reviewing filesUpload.js (a 5,138-line file, unusually large for a single controller), a follow-up scan for any long quoted literal (20+ chars) found ~882 hits that turned out to be 24-character hex MongoDB ObjectIds embedded in a var allOffices = [{ _id: ..., name: ... }, ...] array spanning roughly 4,400 lines — i.e. real office/company records (business names), not Wasabi/Mongo credentials, but real customer data none the less. This is not a credential per the strict definition, but committing ~882 real customer records to source control is its own data-hygiene problem, so it was treated with the same “verify before shipping” discipline.

Traced all usages of allOffices: the only consumer was a helper function, countAllDocs(), that looped over the array and called fileHelper.getAllFilesUpdate() per office — evidently a one-off manual migration/backfill script someone ran by hand at some point. countAllDocs()’s only call site was itself commented out (// countAllDocs();), meaning both the array and the function were unreachable dead code with zero effect on live behavior.

Both were removed before committing (allOffices array + countAllDocs() function), replaced with an explanatory comment. This shrank the committed filesUpload.js from 83,059 bytes / 5,138 lines to 22,597 bytes / 702 lines. Verified node --check passes on the cleaned file and confirmed (via string search) allOffices/countAllDocs have zero remaining references outside the explanatory comments — i.e. this is a no-op from the running application’s point of view, not a behavior change.

Fix applied (this PR)

1. infra-src/pinbox24/w3/persistent-patches/{local.js,controller.js,crudService.js} committed to source control, verbatim (credential-scanned first, zero redactions needed).
2. infra-src/pinbox24/w3/persistent-patches/filesUpload.js committed with the dead allOffices/countAllDocs() block (real customer data, unreachable code) removed — see above. All actual logic (including the 2026-06-30 mirror-insert fix and 2026-07-06 regId/recId fallback) is unchanged.
3. .github/workflows/secrets-sync.yml — added a “Deploy W3 persistent-patches to bms-1” step in sync-pinbox24-w3, run after docker-compose.yml deploy and before container recreation, so this directory is shipped from git on every sync instead of relying on manual docker cp/vi edits.

With this in place, W3’s persistent patches are now visible in source control, reviewable in PRs, and kept in sync with bms-1 by CI on every secrets-sync.yml run — closing the last open item from PR #3248’s follow-up list.

Follow-ups / related open issues (not addressed here)

• pinbox24-w3-operations.md also lists a non-persistent-patches/ volume mount — app-backend/helperFunction/processHelper/tasksService/tasks/excel_import.js — mounted directly from the build-dir source tree (not the persistent-patches/ pattern). RESOLVED — needs no p24-infra source of truth (#4708, verified read-only on bms-1 2026-07-30 by infra-task worker bms4-cw-1). Unlike the 5 persistent-patches/*.js files (which are untracked in the GitLab checkout and delivered by secrets-sync.yml), excel_import.js is a stock, git-tracked file in pinbox24/p24-v-3.2 — it is not a p24-infra-managed hotfix at all:
  • git ls-files --error-unmatch matches; git status --porcelain and git diff --stat HEAD are both empty → tracked and clean, zero local modifications in the live build dir (/home/gitlab-runner/builds/eZQeLfuJe/0/pinbox24/p24-v-3.2, HEAD 9f58b9cb).
  • Last commit touching it is upstream 5f469287 (2020-02-01, “Merge branch ‘feature/prod-docker’”) — the “RESO Excel import fixes” label in pinbox24-w3-operations.md is misleading; there is no divergent patch here.
  • Build-dir copy, mounted copy, and the image’s baked-in copy all hash identically (md5 659f133a695fb3688e8e6f2b78e61d24) → the mount is a redundant no-op. On any CI git clean/re-checkout the file returns automatically with the checkout (it is tracked, not untracked), and even if the mount is lost entirely the container falls back to the byte-identical copy baked into the image. So there is nothing to lose and nothing to mirror into git.
  • The earlier “would require mirroring a much larger portion of the app-backend source tree” concern is therefore moot — no mirroring is needed because GitLab is already the source of truth.
• docs/playbooks/pinbox24-w3-operations.md still lists open issues #3060 (MongoDB admin password exposed in PM2 logs) and #3063 (docker-deploy-prod.sh hardcoded credentials) — unrelated to this fix, tracked separately.