s3-v2-v42-prod — File Service on bms-1

Service: s3-v2-v42-prod Docker container on bms-1 (94.23.26.113) Purpose: File storage microservice for w4.pinbox24.com — stores and serves user-uploaded files Bucket: test-replicated-to-us-bucket on Wasabi EU Central (note: the name looks like a test bucket but is the production bucket) Deploy path: /root/s3v2-prod/ on bms-1 Last updated: 2026-07-01

1. What This Service Is

s3-v2-v42-prod is the file storage microservice for the Pinbox24 w4 product. It:

Accepts file uploads from the main v42-prod API and stores them in Wasabi S3 (test-replicated-to-us-bucket)
Records file metadata in MongoDB (w4_db.files collection, 214K+ documents as of 2026-07-01)
Serves file download requests from w4.pinbox24.com by looking up the file doc in MongoDB and returning the S3 object

The service is managed via docker-compose in /root/s3v2-prod/ on bms-1.

2. Critical Configuration — MONGODB_URI

The MONGODB_URI MUST point to w4_db on the rs0 cluster (bms-2/bms-3), NOT to eat1 on eat-hn.

Parameter	Correct value
Database name	`w4_db`
Cluster	rs0 (bms-2: `145.239.133.104`, bms-3: `51.68.155.224`)
Collection	`files` (214K+ docs)

Wrong configuration (do NOT use):

Database: eat1
Cluster: eat-hn (external hosted cluster)

If MONGODB_URI points to eat1, file lookups will return “File Doc not found in db” for every request because the w4_db.files collection does not exist on eat-hn.

3. File Locations on bms-1

/root/s3v2-prod/
├── docker-compose.yml          # compose definition
├── s3-v2-environment.env       # active env vars (use this for config changes)
└── docker-deploy-prod.sh       # ⚠️ WARNING: contains hardcoded credentials — see §6

Always use s3-v2-environment.env for reading or modifying configuration. Never read docker-deploy-prod.sh — it contains hardcoded credentials (see §6).

Auto-synced by secrets-sync.yml (#3220, #3235). The sync-pinbox24-w4 job now writes MONGODB_URI and the Wasabi credential keys it finds in this file (pinbox24PublicAccessKeyId, pinbox24PublicSecretAccessKey, s3Bucket_api_accessKeyId, s3Bucket_api_secretAccessKey, s3Bucket_BUCKET_NAME, s3Bucket_endpoint) from the corresponding V42_* keys in secrets/pinbox24-w4.env.sops on every W4 sync, then restarts the container. This closes the drift where a Wasabi rotation left s3-v2-v42-prod on stale keys. The sync is update-only — it refreshes keys already present in the file and never appends new ones, so manual additions here are still safe. To change a value permanently, update the SOPS file (do not hand-edit only the live env file, or the next sync will overwrite it).

4. How to Restart the Container

SSH to bms-1 first:

ssh root@94.23.26.113

Then restart:

docker stop s3-v2-v42-prod
docker rm s3-v2-v42-prod
IMAGE_NAME=s3-v2-v42-prod:latest CONTAINER_NAME=s3-v2-v42-prod \
  docker-compose -f /root/s3v2-prod/docker-compose.yml up -d

Verify it came up:

docker ps | grep s3-v2-v42-prod
docker logs --tail 30 s3-v2-v42-prod

5. Diagnosing File Download Failures

Step 1 — Check PM2 logs

ssh root@94.23.26.113
tail -200 /var/log/s3-v2-v42-prod/pm2/out.log | grep -i "error\|not found\|fail"

Step 2 — Look for “File Doc not found in db”

If you see this error, the service is failing to find file records in MongoDB. Most common cause: MONGODB_URI points to the wrong database.

Step 3 — Verify the MONGODB_URI database name (safe, no secrets shown)

docker exec s3-v2-v42-prod env | grep MONGODB_URI | sed 's/.*\///;s/?.*//'

This extracts and prints only the database name from the URI (e.g. w4_db). Expected output: w4_db If it shows eat1 or anything else, the URI is wrong — correct it in s3-v2-environment.env and restart.

Step 4 — Correct the MONGODB_URI if wrong

# On bms-1, edit the env file:
nano /root/s3v2-prod/s3-v2-environment.env
# Change MONGODB_URI to point to w4_db on rs0 cluster
# Then restart (see §4)

The correct URI format (key names only — look up values in secrets/bms-servers.env.sops):

MONGODB_URI=mongodb://<w4_user>:<w4_password>@145.239.133.104:27017,51.68.155.224:27017/w4_db?replicaSet=rs0&authSource=w4_db

Step 5 — Verify Wasabi connectivity

docker exec s3-v2-v42-prod curl -s -o /dev/null -w "%{http_code}" \
  https://s3.eu-central-1.wasabisys.com

Expected: 200 or 403 (403 = reachable, just no auth). Any other code indicates network issue.

6. Security Warning — docker-deploy-prod.sh

/root/s3v2-prod/docker-deploy-prod.sh contains hardcoded credentials. Do NOT run cat, less, head, or any read command on this file.

The following key types are hardcoded in this file (key names only):

pinbox24PublicAccessKeyId / pinbox24PublicSecretAccessKey — Wasabi S3 IAM keys
mailgunPassword / MAILGUN_PASSWORD — Mailgun API key
JWT_TOKEN_SECRET / SESSION_SECRET — JWT signing secrets
monitoringToken — internal monitoring token
PM2_PUBLIC_KEY / PM2_SECRET_KEY — PM2 Plus monitoring keys
MongoDB URI with embedded credentials

These credentials were exposed in a chat transcript on 2026-07-01 (incident tracked in GH issue [Security] Credential exposure — docker-deploy-prod.sh on bms-1).

Required remediation (tracked in the GH security issue)

All credentials must be moved to secrets/bms-servers.env.sops (or a new secrets/pinbox24-bms1.env.sops)
docker-deploy-prod.sh must have hardcoded values replaced with SOPS-sourced references
Rotate all exposed credentials per docs/playbooks/static-api-key-incident-rotation.md

Until remediation is complete, manage env vars exclusively via s3-v2-environment.env.

7. Wasabi Bucket

Parameter	Value
Bucket name	`test-replicated-to-us-bucket`
Region	Wasabi EU Central (`s3.eu-central-1.wasabisys.com`)
Note	Despite the name, this is the production bucket for w4.pinbox24.com files

The bucket name is misleading — it was named when the bucket was created for testing and then promoted to production without renaming.

8. Escalation Path

Symptom	Action
”File Doc not found in db”	Check MONGODB_URI db name (§5). Fix env, restart.
Files upload but can’t download	Check Wasabi IAM key validity. Rotate via Wasabi console or wasabi-iam-rotator.
Container won’t start	Check `docker logs s3-v2-v42-prod`. Usually env file syntax error or MongoDB unreachable.
MongoDB unreachable	Verify rs0 health on bms-2/bms-3. Follow MongoDB rs0 playbook.
Credential rotation needed	Follow `docs/playbooks/static-api-key-incident-rotation.md`. Never read deploy script.

p24-infra Docs

Explorer

s3-v2-bms1-file-service