p24-infra Docs

pinbox24-mailgun-s3v2-stabilization

6 min read

Playbook: Pinbox24 Mailgun → s3-v2 Pipeline Stabilization

Last updated: 2026-07-02
Status: Temporary patch applied — permanent fix pending GitLab MR
Affects: p4-ms-mailgun (bms-1: mailgun-prod_mailgun), pinbox24-ms-s3-v2 (bms-1: s3-v2-v42-prod)

Current state (after 2026-07-02 session)

The pipeline is partially working:

StepStatusNotes
Mailgun webhook → p4-ms-mailgun✅Receives stored email, creates record in v42-prod
Record creation in w4_db.regRecords✅Fields mapped correctly via emailBodyGlobal
Process engine start (aiProcessingLog)✅status fixed from preparation → launched
avoidDupCheck switch (act. 308)✅Routes correctly when ai-email-address is set
check duplicates (act. 296)✅ (partial)mqFilter syntax fixed; returns duplicateCount=0 for new records
Attachment upload to s3-v2✅Route method patched GET→POST (temporary); file stored in Wasabi
recordMainDocument update on record❌Never happens — missing call in mailgunFileHandlerHelper
Process continues past act. 300❌Stuck at “duplicates to cofirm” — duplicate: 'Y' set despite duplicateCount: 0

Temporary patches applied (bms-1, not in source)

1. s3-v2-v42-prod: mailgunFileHandler route method

File: /app/dist/app.routing.js line 65
Change: method: "get" → method: "post"
Applied: docker exec s3-v2-v42-prod sed -i "65s/method: \"get\"/method: \"post\"/" /app/dist/app.routing.js
Lost on: container restart / redeploy
Permanent fix: Change method in the TypeScript source in pinbox24-ms-s3-v2 GitLab repo — see MR spec below.

2. mailgun-prod_mailgun: S3_V2_SERVER_URL env var

File: /root/mailgun-prod/mailgun-environment.env
Change: Added S3_V2_SERVER_URL=http://s3-v2-v42-prod:3000
Applied: echo 'S3_V2_SERVER_URL=http://s3-v2-v42-prod:3000' >> /root/mailgun-prod/mailgun-environment.env
Survives: container restart (env file is read on docker-compose up)
Permanent fix: Add S3_V2_SERVER_URL to docker-deploy-prod.sh in pinbox24/p4-ms-mailgun GitLab repo (when credentials are moved out of that file per issue #2052).

3. w4_db.process.configs (aiProcessingLog): status and mqFilter

Applied directly to MongoDB — these ARE permanent:

  • status: "preparation" → "launched" (process now runs)
  • Activity 296 mqFilter: fileSize=##fileSize## → fileSize='##fileSize##' (quoted, prevents parse error on empty value)

Permanent fixes required (GitLab MRs)

MR 1 — pinbox24-ms-s3-v2: fix mailgunFileHandler route + add recordMainDocument update

File: source file for app.routing.js (TypeScript, likely src/app.routing.ts or similar)

// BEFORE
{
  path: "/api/v3/storage/:officeId/mailgunFileHandler",
  method: "get",   // ← BUG: must be post
  handler: [...]
}
 
// AFTER
{
  path: "/api/v3/storage/:officeId/mailgunFileHandler",
  method: "post",  // ← fix
  handler: [...]
}

File: src/apps/storage/mailgunFileHandler.helper.ts

Add recordMainDocument update after file creation. The reqBody already contains recId and regId. After doc = await FileModel.create(...), call v42-prod to update:

// After: doc = await uploadMultiFiles(key, fileData, fileData.mimeType, doc._id);
// Add:
if (reqBody.recId && doc._id) {
  await mainServerRequest.updateRecord(reqBody.regId, reqBody.recId, {
    recordMainDocument: doc._id.toString()
  });
}

The mainServerRequest helper (or equivalent) must accept a PATCH/PUT to v42-prod’s update record endpoint. Check src/globalHelpers/mainServerRequest.helper.ts for the existing pattern used by other helpers in this service.

MR 2 — pinbox24/p4-ms-mailgun: reorder flow (file-first) + add S3_V2_SERVER_URL

File: docker-deploy-prod.sh
Add: S3_V2_SERVER_URL=http://s3-v2-v42-prod:3000
(After credential rotation from issue #2052 — do NOT add new vars before credentials are moved out)

Architectural change — upload file BEFORE creating register record:

Current broken order: create record → process engine starts (no file data) → upload file (recordMainDocument never set)

Correct order:

  1. Fetch attachment metadata from Mailgun storage URL (name, size, content-type)
  2. Call s3-v2 mailgunFileHandler → creates file doc in MongoDB + uploads to Wasabi → returns { _id, originalName, size }
  3. Call v42-prod to create register record, passing recordMainDocument, originalName, fileSize from step 2
  4. Process engine starts with complete file data → duplicate check can compare real originalName/fileSize values → works correctly

File: src/helper/s3ServerRequest.helper.js
The mailgunFileHandlerHelper on the s3-v2 side must return { _id, originalName, size } (see MR1). The mailgun side currently does return await rp.post(options.url, options) — that return value must then be used to enrich the record creation body.

File: src/routes/integration.js (or wherever actionOnMail is defined)
Reorder the calls:

// 1. Upload file first, get back file metadata
const fileResult = await uploadMailgunAttachments(attachments, {officeId, regId, ...}, senderEmail);
// fileResult = { _id, originalName, size } from s3-v2
 
// 2. Create record with file info pre-filled
const record = await mainServer.createRecord(regId, {
  aiProc: 'pdf',
  aiProcInfo: 'docs_ai.waitingForProcessing',
  costCategory: 'Rechnung',
  recordMainDocument: fileResult._id,
  originalName: fileResult.originalName,
  fileSize: fileResult.size,
  ...emailFields
});
// Process engine now starts with full data — duplicate check works

Why originalName/fileSize not recordMainDocument for dedup: comparing by recordMainDocument._id would never find duplicates (every upload creates a new unique file doc). The correct dedup signal is same filename + same file size = likely the same invoice re-submitted. Keep the existing mqFilter logic; it will work once originalName and fileSize are populated on the record at creation time.

Process engine fixes required (w4_db.process.configs)

Issue: act. 300 “duplicates to confirm” triggered despite duplicateCount: 0

duplicate: 'Y' is being set even though duplicateCount = 0. Investigate the task logic in activities 297–300 to find where duplicate is set and under what condition. Likely a field update task sets duplicate: 'Y' unconditionally before checking duplicateCount.

Correct behavior: when duplicateCount = 0 and originalName is empty (no file yet), skip the duplicate check entirely or continue to the next substantive state (waiting for AI processing).

Issue: duplicate check runs before file is uploaded

Architecture flaw: the process engine starts immediately on record creation, before the attachment is uploaded. Activities 296–300 check for file duplicates by originalName/fileSize, but these fields are always empty at process start.

Proposed fix: Add a switch activity before 296 that checks if originalName is empty (##isEmpty##) and bypasses the duplicate check, routing directly to a “waiting for file” state. The duplicate check should re-run after recordMainDocument is set (either via a process transition triggered by the file upload, or a cron job).

How to re-apply temporary patch after container redeploy

If s3-v2-v42-prod is redeployed before the GitLab MR is merged:

# On bms-1
docker exec s3-v2-v42-prod grep -n "mailgunFileHandler" /app/dist/app.routing.js
# Find the line number with method: "get" for the mailgunFileHandler POST route
# Then patch (replace 65 with actual line number):
docker exec s3-v2-v42-prod sed -i "65s/method: \"get\"/method: \"post\"/" /app/dist/app.routing.js
docker exec s3-v2-v42-prod pm2 restart all

Verify patch applied:

docker exec s3-v2-v42-prod grep -A2 "mailgunFileHandler" /app/dist/app.routing.js | grep method
# Should show: method: "post"

Verification steps (end-to-end test)

After GitLab MRs are deployed:

  1. Send test email with PDF to rechnung-AI@integrations-eu.pinbox24.com
  2. Check PM2 logs: docker exec mailgun-prod_mailgun tail -30 /var/log/mailgun-v42-prod/pm2/pm2_mailgun-v42-prod_production_out.log
    • Expect: POST /api/v1/integration/add 200
  3. Check s3-v2 logs: docker exec v42-prod tail -10 /var/log/s3-v2-v42-prod/pm2/pm2_s3-v2-v42-prod_production_out.log
    • Expect: uploading file started / uploading file ended — no “Method not found”
  4. Check MongoDB: 
    // In mongosh (w4_db):
db.regRecords.findOne({regId: ObjectId("67597cb44eefb1002c9847d4")}, {}, {sort: {createdAt: -1}})
// Expect: recordMainDocument is set to a file _id
db.files.findOne({recId: <record_id>})
// Expect: file document with storageInfo showing Wasabi upload
  5. Confirm process moves past act. 300 to AI processing state

Graph View

Backlinks