Playbook: Wasabi Credentials Broken — radekkonarski-personal-brand

Trigger: Any Wasabi operation in the brand repo fails with InvalidAccessKeyId
Affected scripts: scripts/youtube/upload.py, scripts/publish.py, scripts/youtube/replace_video09.py
Bucket: s3://p24-infra (shared, eu-central-2, Wasabi)
Root cause pattern: WASABI_ACCESS_KEY in radekkonarski-brand.env.sops goes stale when keys are rotated in the shared bucket without syncing to the brand SOPS file


Step 1 — Confirm the error

Run a test from the brand repo:

cd C:\code_2026\radekkonarski-personal-brand
$env:SOPS_AGE_KEY_FILE = "C:\Users\konar\.age\p24-infra-keys.txt"
sops --decrypt --input-type dotenv --output-type dotenv secrets\radekkonarski-brand.env.sops |
  Out-File .env.local -Encoding utf8
python -c "
import os, boto3
from botocore.config import Config
for line in open('.env.local'):
    k,_,v = line.partition('='); os.environ.setdefault(k.strip(), v.strip())
s3 = boto3.client('s3',
    endpoint_url='https://s3.eu-central-2.wasabisys.com',
    aws_access_key_id=os.environ.get('WASABI_ACCESS_KEY',''),
    aws_secret_access_key=os.environ.get('WASABI_SECRET_KEY',''),
    config=__import__('botocore.config',fromlist=['Config']).Config(signature_version='s3v4'),
    verify=False)
r = s3.list_objects_v2(Bucket='p24-infra', Prefix='radekkonarski-brand/', MaxKeys=1)
print('OK')
"
  • OK → credentials still work, wrong diagnosis
  • InvalidAccessKeyId → proceed to Step 2

Step 2 — Find valid credentials

The monitoring stack writes backups to the same p24-infra bucket daily. Its credentials are always current.

$env:SOPS_AGE_KEY_FILE = "C:\Users\konar\.age\p24-infra-keys.txt"
$lines = sops --decrypt --input-type dotenv --output-type dotenv C:\code_2026\p24-infra\secrets\monitoring.env.sops
$m = @{}; foreach ($l in $lines) { if ($l -match '^([^=]+)=(.*)$') { $m[$matches[1]] = $matches[2] } }
$env:TEST_KEY    = $m['WASABI_ACCESS_KEY']
$env:TEST_SECRET = $m['WASABI_SECRET_KEY']
 
python -c "
import os, boto3
from botocore.config import Config
s3 = boto3.client('s3',
    endpoint_url='https://s3.eu-central-2.wasabisys.com',
    aws_access_key_id=os.environ['TEST_KEY'],
    aws_secret_access_key=os.environ['TEST_SECRET'],
    config=Config(signature_version='s3v4'), verify=False)
r = s3.list_objects_v2(Bucket='p24-infra', Prefix='radekkonarski-brand/', MaxKeys=1)
print('monitoring credentials: OK')
"
  • OK → monitoring has the correct key; proceed to Step 3
  • InvalidAccessKeyId again → the key no longer exists in Wasabi at all; skip to Step 5 (create new key)

Step 3 — Update brand SOPS with correct credentials

$env:SOPS_AGE_KEY_FILE = "C:\Users\konar\.age\p24-infra-keys.txt"
cd C:\code_2026\radekkonarski-personal-brand
 
# Decrypt monitoring SOPS → extract Wasabi keys
$monLines = sops --decrypt --input-type dotenv --output-type dotenv C:\code_2026\p24-infra\secrets\monitoring.env.sops
$m = @{}; foreach ($l in $monLines) { if ($l -match '^([^=]+)=(.*)$') { $m[$matches[1]] = $matches[2] } }
 
# Decrypt brand SOPS → replace Wasabi keys
$brandLines = sops --decrypt --input-type dotenv --output-type dotenv secrets\radekkonarski-brand.env.sops
$updated = $brandLines | ForEach-Object {
    if ($_ -match '^WASABI_ACCESS_KEY=') { "WASABI_ACCESS_KEY=$($m['WASABI_ACCESS_KEY'])" }
    elseif ($_ -match '^WASABI_SECRET_KEY=') { "WASABI_SECRET_KEY=$($m['WASABI_SECRET_KEY'])" }
    else { $_ }
}
 
# Write temp file into secrets/ (matches .sops.yaml path_regex)
$tmp = "secrets\brand-edit.env.sops"
[System.IO.File]::WriteAllText("$PWD\$tmp", ($updated -join "`n") + "`n", [System.Text.UTF8Encoding]::new($false))
 
# Re-encrypt
$enc = sops --encrypt --input-type dotenv --output-type dotenv $tmp
[System.IO.File]::WriteAllText("$PWD\secrets\radekkonarski-brand.env.sops", ($enc -join "`n") + "`n", [System.Text.UTF8Encoding]::new($false))
 
Remove-Item $tmp -Force
Write-Host "SOPS updated"

Step 4 — Refresh .env.local and verify

$env:SOPS_AGE_KEY_FILE = "C:\Users\konar\.age\p24-infra-keys.txt"
cd C:\code_2026\radekkonarski-personal-brand
 
$content = sops --decrypt --input-type dotenv --output-type dotenv secrets\radekkonarski-brand.env.sops
[System.IO.File]::WriteAllText("$PWD\.env.local", ($content -join "`n") + "`n", [System.Text.UTF8Encoding]::new($false))
 
# Re-run the test from Step 1 → should print OK

Then commit and push:

git add secrets/radekkonarski-brand.env.sops
git commit -m "secrets: sync Wasabi credentials from monitoring.env.sops"
git push

Step 5 — If monitoring credentials also fail (key deleted in Wasabi)

This requires Radek to create a new sub-user key in the Wasabi console:

  1. Log in to console.wasabisys.com as the p24-infra account owner
  2. Access KeysCreate Access Key for the p24-infra-shared sub-user (or create a new one if deleted)
  3. Copy the new Access Key ID and Secret Key
  4. Update both SOPS files:
    • C:\code_2026\p24-infra\secrets\monitoring.env.sops — update WASABI_ACCESS_KEY / WASABI_SECRET_KEY
    • C:\code_2026\radekkonarski-personal-brand\secrets\radekkonarski-brand.env.sops — same keys
  5. Re-decrypt both repos’ .env.local
  6. Re-run Step 1 verification

Create a GitHub issue with milestone human-action:

gh issue create --repo radieu/p24-infra `
  --title "Wasabi key deleted — brand + monitoring need new access key" `
  --label "human-action,bug" `
  --milestone "human-action" `
  --body "Wasabi InvalidAccessKeyId on both monitoring and brand credentials.
The sub-user key no longer exists in Wasabi console.
Action: create new access key at console.wasabisys.com, then update monitoring.env.sops + radekkonarski-brand.env.sops."

Step 6 — Re-upload pending Wasabi assets

After credentials are fixed, upload any videos that have wasabi_url: null in produced.json:

cd C:\code_2026\radekkonarski-personal-brand
python -c "
import json
p = json.load(open('scripts/heygen/produced.json'))
for k,v in p.items():
    if not v.get('wasabi_url'): print(k, v.get('heygen_url','NO URL'))
"

For each missing video, run:

python scripts/youtube/upload.py --video VIDEO-XX

Or for a one-off replacement (VIDEO-09 pattern):

python scripts/youtube/replace_video09.py

Known incidents

DateKey sourceRoot causeFix
2026-06-21radekkonarski-brand.env.sopsStale key — monitoring was rotated, brand SOPS not updatedCopied valid key from monitoring.env.sops (Step 3)

Prevention

The brand repo’s WASABI_ACCESS_KEY should match the monitoring stack’s key. When rotating Wasabi credentials in monitoring.env.sops, always update radekkonarski-brand.env.sops in the same commit.