sops-edit-operations

Playbook: SOPS Secret File Edit Operations

Root cause of “no matching creation rules found”

SOPS path_regex in .sops.yaml matches against the input file path when encrypting, not the --output destination. The --age flag overrides key recipients but does not bypass the path_regex validation — SOPS still requires a matching creation rule even when keys are given explicitly.

# .sops.yaml
creation_rules:
  - path_regex: (^|[/\\])secrets[/\\].*\.env\.sops$

Temp file name	Matches regex?	Result
secrets/monitoring-edit.env	NO	error: no matching creation rules found
secrets/monitoring-edit.env.sops	YES	encrypts correctly

Mental model: SOPS encrypts files into a format, and the creation rule determines which keys to use based on “what kind of file is this going to be” — judged by the source path.

---

Standard pattern for editing a SOPS secrets file

$env:SOPS_AGE_KEY_FILE = "C:\Users\konar\.age\p24-infra-keys.txt"
$SOPS_FILE = "C:\code_2026\p24-infra\secrets\monitoring.env.sops"
$TEMP_FILE = "C:\code_2026\p24-infra\secrets\monitoring-edit.env.sops"  # ← must match path_regex
 
# 1. Decrypt to temp (name MUST end in .env.sops to match path_regex)
$plain = sops --decrypt --input-type dotenv --output-type dotenv $SOPS_FILE
[System.IO.File]::WriteAllText($TEMP_FILE, ($plain -join "`n") + "`n", [System.Text.UTF8Encoding]::new($false))
 
# 2. Edit: append, replace, or remove keys — ALWAYS via ReadAllText/WriteAllText (never Add-Content / Set-Content)
$txt = [System.IO.File]::ReadAllText($TEMP_FILE)
$txt = $txt.TrimEnd("`n") + "`nNEW_KEY=value`n"   # append
[System.IO.File]::WriteAllText($TEMP_FILE, $txt, [System.Text.UTF8Encoding]::new($false))
# To replace an existing value:
$txt = [System.Text.RegularExpressions.Regex]::Replace($txt, '(?m)^OLD_KEY=.*$', 'OLD_KEY=newval')
[System.IO.File]::WriteAllText($TEMP_FILE, $txt, [System.Text.UTF8Encoding]::new($false))
 
# 3. Re-encrypt (TEMP_FILE matches path_regex → creation rule found → success)
sops --encrypt --input-type dotenv --output-type dotenv --output $SOPS_FILE $TEMP_FILE
$exitCode = $LASTEXITCODE
 
# 4. ALWAYS clean up plaintext — even on failure
Remove-Item $TEMP_FILE -Force -ErrorAction SilentlyContinue
if ($exitCode -ne 0) { throw "SOPS encrypt failed — original file unchanged" }

Why --output flag alone is not enough

# WRONG: --output only changes the destination; SOPS still reads input path for creation rules
sops --encrypt ... --output secrets/monitoring.env.sops secrets/monitoring-edit.env
# → error: no matching creation rules found  (because input doesn't match regex)
 
# RIGHT: input file path must match the regex
sops --encrypt ... --output secrets/monitoring.env.sops secrets/monitoring-edit.env.sops
# → success

---

Critical: --in-place requires explicit --input-type dotenv --output-type dotenv

SOPS 3.9.1 bug: sops --encrypt --in-place file.env.sops without explicit types outputs JSON format even for .env.sops files. The JSON output is unreadable by sops --decrypt --input-type dotenv → “invalid dotenv input line: {”.

# WRONG — produces JSON output that cannot be decrypted as dotenv
sops --encrypt --in-place secrets/monitoring-edit.env.sops
 
# CORRECT — explicit types preserve dotenv format
sops --encrypt --input-type dotenv --output-type dotenv --in-place secrets/monitoring-edit.env.sops

Always verify with canary decrypt after encryption.

---

Critical: sops updatekeys is unusable on dotenv files — no flag fixes it

Never run sops updatekeys against a secrets/*.env.sops file. It always fails:

Error unmarshalling input json: invalid character 'C' looking for beginning of value

(the reported character is simply the first character of the first key in the file — 'C' for a file starting with CF_AP…).

Why — and why this one is worse than --encrypt --in-place

updatekeys selects its store from the file extension alone and ignores --input-type entirely. Our dotenv secrets are named *.env.sops, so the effective extension is .sops — not a store SOPS knows — and it falls back to the JSON store, which then tries to JSON-parse the encrypted dotenv body and dies on its first character.

This is the same bug class as --encrypt --in-place above, but that one honours explicit --input-type/--output-type, so it has a flag-based fix. updatekeys does not plumb the flag through at all — no flag combination makes it work.

Verified empirically on SOPS 3.9.1 against a throwaway age keypair + throwaway dotenv fixture (issue #4601 — never against a real secrets file):

Command	Result
sops updatekeys --yes secrets/x.env.sops	exit 1 — Error unmarshalling input json: invalid character …
sops updatekeys --yes --input-type dotenv --output-type dotenv secrets/x.env.sops	exit 1 — flag provided but not defined: -output-type. The flag does not exist on this subcommand
sops updatekeys --yes --input-type dotenv secrets/x.env.sops	exit 1 — identical JSON-unmarshal error. --input-type is accepted but ignored
Setting input_type: dotenv / output_type: dotenv on the matching .sops.yaml creation_rule	exit 1 — no effect; creation-rule types apply at encrypt time, not to updatekeys
Byte-identical encrypted content renamed to secrets/x.env (.env extension)	exit 0 — recipients synced, canary decrypt OK. Confirms the extension is the whole trigger

Non-destructive. The failure happens before any write: after a failed run the file is byte-identical (same md5), the recipient count is unchanged, and canary decrypt still succeeds. The cost is a stalled recipient rotation, not data loss — but a rotation that silently stops half-way across secrets/*.env.sops leaves an inconsistent recipient set, which is why this is worth guarding rather than just retrying.

Correct procedure — no-op key/value rewrite

Re-encrypting the file with no value changes achieves exactly what updatekeys is meant to do: Write-SopsFileContent resolves the creation_rule fresh from .sops.yaml and encrypts to whatever recipient set that rule currently lists — plus the safety net updatekeys never had (temp-file staging, canary decrypt, recipient-count verification, atomic move, automatic rollback).

$env:SOPS_AGE_KEY_FILE = "C:\Users\konar\.age\p24-infra-keys.txt"
 
# One file — preferred, single command:
.\scripts\sops-set.ps1 -SopsFile secrets\monitoring.env.sops -RekeyOnly
 
# Every file, after editing the recipients in .sops.yaml:
Get-ChildItem secrets\*.env.sops | ForEach-Object {
    .\scripts\sops-set.ps1 -SopsFile $_.FullName -RekeyOnly
}

Or directly, from a session that has already imported the module:

Import-Module .\scripts\lib\sops-common.psm1 -Force
Update-SopsKeys -SopsFile secrets\monitoring.env.sops -Pairs @{}   # empty hashtable = no-op rewrite

-RekeyOnly reports the resulting recipient count. If it does not match the number of age: entries on the matching .sops.yaml creation_rule, Write-SopsFileContent refuses the write and restores the original — so a silently over- or under-widened recipient set cannot ship.

sops-set.ps1 -RekeyOnly is the switch to use. Passing -Pairs '{}' does not work — the CLI rejects an empty batch with “-Pairs parsed to zero key/value entries” (that guard exists to catch a malformed JSON payload, and is deliberately left in place).

This is enforced by a hook. .claude/hooks/pre-bash-safety.sh and pre-bash-safety-windows.ps1 block any sops … updatekeys … command naming a .env.sops file, the same way they already block hand-rolled sops --encrypt calls. Regression tests: tests/hooks/test_pre_bash_safety.py. updatekeys on a non-dotenv SOPS file (.yaml/.json) is unaffected and stays permitted — this bug is dotenv-specific.

---

Linux / vps-i1: use Python to avoid shell quoting issues

When editing a SOPS dotenv file via SSH from Windows, PowerShell heredoc quoting breaks sed commands (caret ^ in regex, pipe | in sed). Use a Python script instead:

#!/usr/bin/env python3
import subprocess, os, shutil
 
SOPS_FILE = '/opt/p24-infra/secrets/monitoring.env.sops'
TMP_ENC   = '/opt/p24-infra/secrets/monitoring-upd.env.sops'
env = os.environ.copy()
env['SOPS_AGE_KEY_FILE'] = '/home/claude-runner/.age/p24-infra-keys.txt'
 
# 1. Decrypt
r = subprocess.run(['sops','--decrypt','--input-type','dotenv','--output-type','dotenv',SOPS_FILE],
    capture_output=True, text=True, env=env, cwd='/opt/p24-infra')
assert r.returncode == 0
 
# 2. Modify (pure Python, no sed quoting issues)
new_lines = []
for line in r.stdout.splitlines(keepends=True):
    if line.startswith('MY_KEY='):
        new_lines.append(f'MY_KEY={new_value}\n')
    else:
        new_lines.append(line)
 
# 3. Write to secrets/ (matches path_regex)
with open(TMP_ENC, 'w', newline='\n') as f:
    f.write(''.join(new_lines))
 
# 4. Encrypt WITH explicit types
r2 = subprocess.run(
    ['sops','--encrypt','--input-type','dotenv','--output-type','dotenv','--in-place', TMP_ENC],
    capture_output=True, text=True, env=env, cwd='/opt/p24-infra')
assert r2.returncode == 0
 
# 5. Canary decrypt
r3 = subprocess.run(['sops','--decrypt','--input-type','dotenv','--output-type','dotenv',TMP_ENC],
    capture_output=True, text=True, env=env)
assert r3.returncode == 0  # if fails, delete TMP_ENC and abort
 
# 6. Replace original
shutil.move(TMP_ENC, SOPS_FILE)

SCP the script to /tmp/, run via SSH, delete after. Never embed the new secret value in the script — read it from a separately SCP’d temp file and wipe immediately after reading.

---

Known broken patterns in SOPS 3.9.1 (dotenv files)

Pattern	Error	Workaround
sops exec-env file.env.sops 'cmd'	”Error unmarshalling input json: invalid character ‘C‘“	Use decrypt → modify → encrypt
sops exec-file file.env.sops ...	Same JSON parse error	Same
sops --set '["KEY"]' '"val"'	”Value for —set is not valid JSON”	Same
sops set file '["KEY"]' '"val"'	”Invalid set index format”	Same
sops --encrypt --in-place without --input-type dotenv	Produces JSON, canary fails	Add explicit --input-type dotenv --output-type dotenv
sops updatekeys [--yes] file.env.sops	”Error unmarshalling input json: invalid character ‘C‘“	No flag fixes this — --input-type is ignored, --output-type does not exist. Use sops-set.ps1 -RekeyOnly (see the dedicated section above)

Root cause: SOPS 3.9.1 edit/set modes try to JSON-parse the encrypted dotenv file before decryption. First char ‘C’ (from e.g. CF_AP… key) triggers “invalid character ‘C’”. For updatekeys specifically the store is chosen from the file extension (.sops → unknown → JSON fallback), which is why renaming the same content to *.env makes it succeed.

---

Confirm fix worked

$env:SOPS_AGE_KEY_FILE = "C:\Users\konar\.age\p24-infra-keys.txt"
sops --decrypt --input-type dotenv --output-type dotenv secrets/monitoring.env.sops |
  Select-String "YOUR_NEW_KEY" |
  ForEach-Object { $_.Line.Split('=')[0] + "=<redacted>" }

---

Escalation

If SOPS consistently fails after following this pattern:

1. Check that $env:SOPS_AGE_KEY_FILE points to the correct age key file
2. Verify the key file contains the developer private key (not just the public key)
3. Run sops --version to check for version regressions
4. Create GitHub issue in radieu/p24-infra with bug label

Prevention

• Claude hooks will catch this error pattern automatically and log it
• The SOPS temp file naming rule is enforced in the error patterns file: .claude/error-patterns.json → sops_no_creation_rule

Audit Log — Log to infra_operations

Editing a SOPS secrets file is a credential operation and MUST be recorded in infra_operations after the canary decrypt passes and the change is committed. Reference the key NAME(s) only — never the value. See docs/infra-operations-audit-operations.md.

# Linux / VPS worker — after canary decrypt succeeds and the change is committed:
source /opt/p24-infra/scripts/lib/log_op.sh
log_op "claude" "credential_rotation" "monitoring.env.sops" "success" \
  "Edited SOPS file — updated KEY_NAME (value not shown); canary decrypt OK" "local" 2730
# op_type: use "credential_rotation" when the edit rotates a secret, "config_change" for a
#          structural edit (adding/removing a key with no live rotation).
# result:  "success" after canary passes; "failed" if encrypt/canary failed.

# Windows dev session — no bash helper; POST directly (continue-on-error, fail-open):
python3 -c "import os,json,urllib.request,ssl; url=os.environ['SUPABASE_URL'].rstrip('/'); key=os.environ['SUPABASE_SERVICE_KEY']; d=json.dumps({'actor':'radieu','op_type':'credential_rotation','resource':'monitoring.env.sops','result':'success','detail':'SOPS edit — KEY_NAME (value not shown); canary OK','env':'local','gh_issue':2730}).encode(); req=urllib.request.Request(f'{url}/rest/v1/infra_operations',data=d,method='POST',headers={'Content-Type':'application/json','apikey':key,'Authorization':f'Bearer {key}','Prefer':'return=minimal'}); urllib.request.urlopen(req,timeout=3,context=ssl.create_default_context())"

The log_op wrapper redacts secret-looking detail fields automatically, but do not rely on it — name the key, never quote the value.

---

Related

• Rotation & batch multi-key writes (decrypt → edit-all → encrypt → canary in one cycle) and why sops set is unusable on dotenv files: sops-rotation-classifier-patterns.md.
• CRLF/BOM corruption (including CR baked into encrypted values): sops-windows-crlf.md.