p24-infra Docs

pinbox24-wkhtml-network-isolation

3 min read

Playbook: Pinbox24 wkhtml Network Isolation — DNS + Routing Fix

Created: 2026-07-07
Incident: w4-pinbox24-staging dispatch book generation fails with Error: getaddrinfo EAI_AGAIN wkhtml-v42-prod
Issue: #3191
Root Cause: Docker network misconfiguration — wkhtml container isolated on prod-v-4-net, unreachable from test-net

Trigger Symptoms

  • API call to generate “księga nadawcza” (dispatch book) fails
  • Error: Error: getaddrinfo EAI_AGAIN wkhtml-v42-prod or ECONNREFUSED
  • Endpoint: /api/reg/postBook/{id}/generate
  • Occurs in both staging (v42-stage) and production (v42-prod) when they try to call wkhtml PDF service
  • Other API endpoints work normally

Root Cause

wkhtml-v42-prod (the wkhtmltopdf-as-a-service container) was only connected to the prod-v-4-net Docker network (172.18.x.x), while:

  • Production v42-prod — on prod-v-4-net (could reach wkhtml)
  • Staging v42-stage — on test-net (172.20.x.x, could NOT reach wkhtml)

Even though Docker DNS could resolve wkhtml-v42-prod as a hostname, the containers had no network path between networks, causing connection timeouts and EAI_AGAIN errors.

Diagnosis (Quick Check)

# SSH to bms-1
ssh root@94.23.26.113
 
# Check what networks wkhtml is on
docker inspect wkhtml-v42-prod --format '{{range $k,$v := .NetworkSettings.Networks}}{{println $k}}{{end}}'
# Expected (before fix): prod-v-4-net only
# Expected (after fix): prod-v-4-net + test-net
 
# Check what network v42-stage is on
docker inspect v42-stage --format '{{range $k,$v := .NetworkSettings.Networks}}{{println $k}}{{end}}'
# Expected: test-net
 
# Verify connectivity from staging container
docker exec v42-stage curl -s http://wkhtml-v42-prod/ | head -5
# Expected after fix: HTTP response (may be 500 if service has bugs, but connection works)

Immediate Fix (Temporary — survives container restart, NOT redeployment)

On bms-1:

docker network connect test-net wkhtml-v42-prod

Verify both networks are now connected:

docker inspect wkhtml-v42-prod --format '{{range $k,$v := .NetworkSettings.Networks}}{{println $k}}{{end}}'
# Output:
# prod-v-4-net
# test-net

Test from staging container:

docker exec v42-stage wget -q -O- http://wkhtml-v42-prod/ | head -5

Permanent Fix (Survives Redeployment via Docker Compose)

The wkhtml-v42-prod service is now defined in the w4 docker-compose with both networks:

File: infra-src/pinbox24/w4/docker-compose.yml

wkhtml:
  container_name: wkhtml-$CONTAINER_NAME
  image: openlabs/docker-wkhtmltopdf-aas:latest
  restart: unless-stopped
  networks:
    - prod-v-4-net
    - test-net
  expose:
    - "80"

When the w4 stack is next redeployed (via CI or manual docker-compose up), wkhtml-v42-prod will automatically:

  1. Start with both networks connected
  2. Be reachable from both production and staging tiers
  3. Persist across server restarts (due to restart: unless-stopped)

To redeploy now without waiting for CI:

cd /root/builds/7N4sbbrB/0/pinbox24/p24-back-ts
CONTAINER_NAME=v42-prod docker-compose -f /path/to/w4/docker-compose.yml up -d wkhtml

Known Limitation

The wkhtmltopdf service may return HTTP 500 (Python/Gunicorn error) even after the network fix. This is a separate issue in the wkhtmltopdf application code, not a network problem. The network isolation fix unblocks the API from reaching the service; any application-level errors in the PDF generation are handled separately.

Prevention

Design rule: Microservices that are called by API backends in multiple tiers (production + staging) must be connected to all networks where those tiers run. In the p24-infra setup:

  • v42-prod (production) runs on prod-v-4-net
  • v42-stage (staging) runs on test-net
  • Any shared service they both call (wkhtml, pdf-gen, etc.) must be on prod-v-4-net + test-net

For new microservices: declare all required networks in docker-compose from day one. Use a checklist:

[ ] Service is on prod network (prod-v-4-net)?
[ ] If called by staging tier, also on test-net?
[ ] Verified connectivity from all calling tiers before production deployment?
  • pinbox24-w3-w4-outage-diagnosis.md — general w3/w4 outage diagnosis (nginx-proxy, MongoDB, RabbitMQ)
  • bms1-post-restart-recovery.md — post-restart container startup procedure (includes wkhtml-v42-prod in step 3)
  • docs/servers/p4-ovh-bms-1-ns367522-operations.md — general bms-1 operations (container inventory, networks)

Graph View