p24-infra Docs

caddy-basicauth-bcrypt-docker-compose

3 min read

Caddy basicauth: bcrypt hash breaks in Docker Compose env_file

Trigger

Caddy crash-loops with:

Error: loading initial config: provision http.authentication.providers.http_basic:
account 0: username and password are required

When basicauth uses {env.STATUS_BASIC_AUTH_USER} / {env.STATUS_BASIC_AUTH_HASH} in Caddyfile.

Root cause

Docker Compose env_file: performs variable substitution on values. Bcrypt hashes contain $ signs (e.g. $2a$14$ABC...). Docker Compose expands $ABC as an undefined variable → empty string. The hash gets garbled. Caddy sees an empty hash and reports “username and password are required”.

docker run --env-file does NOT interpolate — the discrepancy makes debugging confusing.

Confirm it

# Check Caddy status
cd /opt/p24-infra/monitoring && docker compose ps caddy
 
# Check logs
docker compose logs caddy --tail=5 2>&1 | grep -v 'variable is not set'

If you see “username and password are required” AND warnings like:

The "SOME_VAR" variable is not set. Defaulting to a blank string.

where SOME_VAR looks like part of a bcrypt hash — this is the root cause.

Fix

Step 1 — Generate new password and hash

# On vps-i1 or any host with Docker
PASS=$(tr -dc 'A-Za-z0-9' < /dev/urandom | head -c 20)
HASH=$(docker run --rm caddy:2.11.4-alpine caddy hash-password --plaintext "$PASS")
echo "PASS=$PASS"
echo "HASH=$HASH"

Step 2 — Store in SOPS with $$ escaping

On developer workstation (PowerShell):

$env:SOPS_AGE_KEY_FILE = "C:\Users\konar\.age\p24-infra-keys.txt"
 
# Decrypt
$lines = sops --decrypt --input-type dotenv --output-type dotenv secrets\monitoring.env.sops
 
# Escape hash: replace each $ with $$
$escapedHash = $HASH.Replace('$', '$$')
 
# Update lines and re-encrypt
$updated = $lines | ForEach-Object {
    if ($_.StartsWith('STATUS_BASIC_AUTH_HASH=')) { "STATUS_BASIC_AUTH_HASH=$escapedHash" }
    elseif ($_.StartsWith('STATUS_BASIC_AUTH_PASSWORD=')) { "STATUS_BASIC_AUTH_PASSWORD=$PASS" }
    else { $_ }
}
[System.IO.File]::WriteAllText(
    "$PWD\secrets\monitoring-edit.env.sops",
    [string]::Join("`n", $updated) + "`n",
    [System.Text.UTF8Encoding]::new($false)
)
$enc = sops --encrypt --input-type dotenv --output-type dotenv secrets\monitoring-edit.env.sops
[System.IO.File]::WriteAllText(
    "$PWD\secrets\monitoring.env.sops",
    [string]::Join("`n", $enc) + "`n",
    [System.Text.UTF8Encoding]::new($false)
)
[System.IO.File]::Delete("$PWD\secrets\monitoring-edit.env.sops")

Step 3 — Deploy to server

If via secrets-sync (PR + merge → CI deploys automatically), skip to Step 4.

For immediate fix — deploy the .env directly:

# Decrypt, escape hash in the deploy file, SCP
$decrypted = sops --decrypt --input-type dotenv --output-type dotenv secrets\monitoring.env.sops
$tempLines = $decrypted | ForEach-Object {
    if ($_.StartsWith('STATUS_BASIC_AUTH_HASH=')) {
        # Already stored with $$ in SOPS, but verify before SCP
        $_
    } else { $_ }
}
[System.IO.File]::WriteAllText("$env:TEMP\mon.env", [string]::Join("`n", $tempLines) + "`n", [System.Text.UTF8Encoding]::new($false))
scp -i C:\Users\konar\.ssh\id_ed25519 "$env:TEMP\mon.env" "root@217.154.82.162:/opt/p24-infra/monitoring/.env"
Remove-Item "$env:TEMP\mon.env"

Step 4 — Restart Caddy (force-recreate required)

# --force-recreate picks up updated env vars from .env
cd /opt/p24-infra/monitoring
docker compose up -d --no-deps --force-recreate caddy
sleep 4
docker compose ps caddy

docker compose restart caddy does NOT re-read the .env — must use --force-recreate.

Verify

docker compose ps caddy
# Status must be: Up N seconds (healthy)
# NOT: Restarting
 
docker compose logs caddy --tail=5 2>&1 | grep -v 'variable is not set'
# Must NOT contain "username and password are required"

Escalation

If Caddy still fails after above:

  1. Check the hash format in the .env on the server: grep STATUS_BASIC_AUTH_HASH /opt/p24-infra/monitoring/.env
    • Should start with $$2a$$14$$
  2. Verify Docker Compose passes correct value: docker run --rm --env-file /opt/p24-infra/monitoring/.env alpine printenv STATUS_BASIC_AUTH_HASH
    • Will show $$2a$$14$$... (raw, no interpolation) — this is expected
    • What matters is what docker compose passes, not docker run --env-file
  3. Manually test hash: echo "p24admin:$(docker compose exec caddy caddy environ | grep STATUS_BASIC_AUTH_HASH)" | caddy hash-password --verify

Prevention

  • Always escape $ as $$ in .env file values that contain $ signs before Docker Compose reads them
  • When storing bcrypt hashes in SOPS for Docker Compose env_file deployment: escape $ → $$ at SOPS store time
  • The secrets-sync workflow deploys SOPS values verbatim — escaping must be in SOPS, not in the workflow

Graph View