Playbook: Brand Credential Rotation (radekkonarski-personal-brand)

Trigger: Telegram alert from n8n Brand — Credential Health Check (every Monday 08:30 Warsaw) OR BrandCredentialFailed / BrandCredentialRotationOverdue Prometheus alert fires in Grafana.

Runbook version: 1.0 — created 2026-06-23 (issue #997)

What credentials exist

All credentials for the radekkonarski-personal-brand project live in:

secrets/radekkonarski-brand.env.sops   (in p24-infra repo)

Variable	Service	Type	Rotation frequency
`RADEK_BRAND_LINKEDIN_ACCESS_TOKEN`	LinkedIn	OAuth 2.0 access token	~60 days (expires automatically)
`RADEK_BRAND_LINKEDIN_REFRESH_TOKEN`	LinkedIn	OAuth 2.0 refresh token	~365 days from last auth
`RADEK_BRAND_LINKEDIN_CLIENT_ID`	LinkedIn	App client ID	On app rotation / revocation
`RADEK_BRAND_LINKEDIN_CLIENT_SECRET`	LinkedIn	App client secret	365 days
`RADEK_BRAND_HEYGEN_API_KEY`	HeyGen	API key	365 days
`RADEK_BRAND_MAILERLITE_API_KEY`	MailerLite	API key	365 days
`RADEK_BRAND_ANTHROPIC_API_KEY`	Anthropic	API key	180 days
`RADEK_BRAND_WASABI_ACCESS_KEY`	Wasabi S3	IAM access key	180 days
`RADEK_BRAND_WASABI_SECRET_KEY`	Wasabi S3	IAM secret key	180 days (paired with above)
`RADEK_BRAND_YOUTUBE_CLIENT_ID`	YouTube / Google	OAuth 2.0 client ID	On app rotation
`RADEK_BRAND_YOUTUBE_CLIENT_SECRET`	YouTube / Google	OAuth 2.0 client secret	365 days

How to confirm a credential alert

1. Check Telegram alert message

The n8n workflow sends:

⚠️ Brand Credential Check FAILED
2/4 API checks failed:
  • LinkedIn userinfo: HTTP 401
  • HeyGen quota: HTTP 403

HTTP status meanings:

401 Unauthorized — token expired or revoked (rotate immediately)
403 Forbidden — token valid but scope/quota issue (check service dashboard)
429 Too Many Requests — rate limit (wait and retry — not a rotation issue)
5xx — service outage (wait and retry before rotating)

2. Check Grafana

Dashboard: radekkonarski — Brand Pipeline Health URL: https://grafana.vps-i1.infra.zintegrowana.online/d/radekkonarski-brand

Section: “API Credential Live Checks” — red tile = failed service.

3. Check Prometheus alert

ssh root@217.154.82.162 "curl -s http://localhost:9090/api/v1/alerts | python3 -c \"import json,sys; [print(a['labels']['alertname'], a['labels'].get('service','')) for a in json.load(sys.stdin)['data']['alerts'] if 'Brand' in a['labels']['alertname']]\""

Response procedure

Step 1: Identify failing service

From the Telegram message or Grafana panel.

Step 2: Check credential source

$env:SOPS_AGE_KEY_FILE = "C:\Users\konar\.age\p24-infra-keys.txt"
sops --decrypt --input-type dotenv --output-type dotenv C:\code_2026\p24-infra\secrets\radekkonarski-brand.env.sops | Select-String "RADEK_BRAND"
# Do NOT paste the values in chat — just confirm they are present

Step 3: Rotate the failing credential (see per-service procedures below)

Step 4: Update SOPS file with new credential

$env:SOPS_AGE_KEY_FILE = "C:\Users\konar\.age\p24-infra-keys.txt"
# Decrypt to temp file
sops --decrypt --input-type dotenv --output-type dotenv C:\code_2026\p24-infra\secrets\radekkonarski-brand.env.sops |
  Out-File C:\code_2026\p24-infra\secrets\brand-edit.env.sops -Encoding utf8
 
# Edit the temp file (update the specific key)
# Then re-encrypt:
$enc = sops --encrypt --input-type dotenv --output-type dotenv C:\code_2026\p24-infra\secrets\brand-edit.env.sops
[System.IO.File]::WriteAllText(
  "C:\code_2026\p24-infra\secrets\radekkonarski-brand.env.sops",
  ($enc -join "`n") + "`n",
  [System.Text.UTF8Encoding]::new($false)
)
Remove-Item C:\code_2026\p24-infra\secrets\brand-edit.env.sops -Force

Step 5: Deploy to bms-4

After committing and merging to dev, the secrets-sync.yml workflow auto-deploys.

For immediate deployment:

ssh root@54.36.123.110 "grep RADEK_BRAND /opt/p24-infra/bms-4/.env | wc -l"
# Then push new env and restart n8n:
ssh root@54.36.123.110 "cd /opt/p24-infra/bms-4 && docker compose restart n8n-main n8n-worker1 n8n-worker2 n8n-worker3"

Step 6: Verify by re-running the n8n workflow

In n8n on bms-4 (https://n8n.bms-4.infra.zintegrowana.online):

Open workflow: Brand — Credential Health Check (Weekly)
Click “Execute workflow” → all services should show green (HTTP 2xx)

Step 7: Confirm Pushgateway updated

ssh root@217.154.82.162 "curl -s http://localhost:9091/metrics | grep brand_credential_status"
# All values should be 1

Step 8: Update secrets-rotation-log.md

Append a row:

| 2026-06-23 | RADEK_BRAND_LINKEDIN_ACCESS_TOKEN | expired (401) | radieu | yes |

LinkedIn rotation (most critical — 60-day expiry)

LinkedIn access tokens expire every ~60 days. The refresh token lasts ~365 days.

Option A: Use refresh token to get new access token (preferred)

# Run on a machine with the brand repo and LinkedIn credentials
cd path/to/radekkonarski-personal-brand
python scripts/linkedin_auth.py --refresh
# Outputs new RADEK_BRAND_LINKEDIN_ACCESS_TOKEN and its expiry date

Then update secrets/radekkonarski-brand.env.sops with the new token (see Step 4 above).

Option B: Full OAuth re-authorization (if refresh token also expired)

1. Run:  python scripts/linkedin_auth.py
2. Visit the printed authorization URL in a browser
3. Grant access — LinkedIn redirects to the callback URL
4. Copy the authorization code from the redirect URL
5. Paste into the script prompt
6. Script outputs new access + refresh tokens

Update both RADEK_BRAND_LINKEDIN_ACCESS_TOKEN and RADEK_BRAND_LINKEDIN_REFRESH_TOKEN in SOPS.

LinkedIn token expiry monitoring: See Grafana panel “LinkedIn Token — Days Until Expiry” on the radekkonarski-brand dashboard. Alert fires when p24_credential_days_overdue{credential=~".*linkedin.*"} > 0.

Key rotation playbook: docs/workbooks/internal/key-rotation/README.md in the radekkonarski-personal-brand repo.

HeyGen API key rotation

1. Log into https://app.heygen.com → Settings → API
2. Regenerate API key
3. Copy new key
4. Update RADEK_BRAND_HEYGEN_API_KEY in secrets/radekkonarski-brand.env.sops (see Step 4)
5. Re-run brand credential health check to confirm

MailerLite API key rotation

1. Log into https://dashboard.mailerlite.com → Integrations → API
2. Generate new API key
3. Copy new key
4. Update RADEK_BRAND_MAILERLITE_API_KEY in secrets/radekkonarski-brand.env.sops
5. Re-run brand credential health check to confirm

Anthropic API key rotation

1. Log into https://console.anthropic.com → API Keys
2. Create new key → copy value
3. Update RADEK_BRAND_ANTHROPIC_API_KEY in secrets/radekkonarski-brand.env.sops
4. Revoke old key from Anthropic console
5. Re-run brand credential health check to confirm

Escalation path

Situation	Action
HTTP 401/403 on LinkedIn — refresh token works	Rotate access token (Option A)
HTTP 401/403 on LinkedIn — refresh token expired	Full re-auth (Option B) — takes ~5 min
HTTP 401/403 on HeyGen — key invalid	Rotate HeyGen API key
HTTP 5xx on any service	Service outage — wait 30 min and retry before rotating
Pushgateway not receiving metrics	Check n8n workflow on bms-4 is active and has correct BRAND_TELEGRAM_CHAT_ID + service env vars
Alert fires but n8n shows all green	Stale Pushgateway data — check metric age in Grafana “Time Since Last Credential Check” panel

Prevention

The n8n Brand — Credential Health Check workflow runs every Monday 08:30 Warsaw
Alertmanager fires BrandCredentialFailed if any check fails for > 1 hour
The credential-exporter tracks p24_credential_days_overdue for scheduled rotation due dates
LinkedIn token should be refreshed before it expires (set next_due to 50 days after last rotation)
Check the Grafana dashboard weekly after the Monday n8n run to confirm all checks pass

Audit Log — Log to infra_operations

After this operation completes, log it to the infra_operations audit table.

Python (Linux server — bms-4, vps-i1, vps-h1, or similar):

import sys
sys.path.insert(0, '/opt/p24-infra')
from scripts.lib.log_op import log_op
 
log_op(
    actor="claude",  # "radieu" for manual human ops, "claude" for agent
    op_type="credential_rotation",
    resource="brand-credentials",
    result="success",  # "success" | "failed" | "skipped"
    detail="Scheduled rotation — brand API credentials updated in SOPS and deployed",
    env="bms-4",
    gh_issue=2730,
)

PowerShell (Windows dev machine):

$env:SUPABASE_URL = (Get-Content "C:\code_2026\p24-infra\.env.local" | Select-String "^SUPABASE_URL=").ToString().Split("=",2)[1].Trim()
$env:SUPABASE_SERVICE_KEY = (Get-Content "C:\code_2026\p24-infra\.env.local" | Select-String "^SUPABASE_SERVICE_KEY=").ToString().Split("=",2)[1].Trim()
python -c "
import os, sys
sys.path.insert(0, 'C:/code_2026/p24-infra')
from scripts.lib.log_op import log_op
log_op('claude', 'credential_rotation', 'brand-credentials', 'success', 'Scheduled rotation — brand API credentials updated in SOPS and deployed', 'bms-4')
"
$env:SUPABASE_URL = ''; $env:SUPABASE_SERVICE_KEY = ''

p24-infra Docs

Explorer

brand-credential-rotation