p24-infra Docs

gh-secret-set-powershell-pipe-corruption

4 min read

Playbook: gh secret set via PowerShell pipe silently corrupts the value

Symptom: a GH Actions secret set from Windows PowerShell via $env:VAR | gh secret set NAME looks correct (command succeeds, gh secret list shows an updated timestamp), but consumers of the secret fail in ways that point to a malformed value — e.g. curl exits with code 3 (“URL using bad/illegal format or missing URL”) even though the source value is a well-formed URL.

Root cause: PowerShell’s pipeline reformats a string object before handing it to a native executable’s stdin. Piping $env:VAR | native.exe (or any bare string variable) prepends a UTF-8 BOM and appends a trailing CRLF. gh secret set stores those extra bytes as part of the secret value — the BOM/CRLF become literal leading/trailing characters in what the workflow later reads from secrets.NAME.

Confirmed by byte-length test: a 121-character ASCII URL became 126 bytes after $env:VAL | sort.exe > file — exactly BOM (3 bytes) + CRLF (2 bytes) = 5 extra bytes.

This is not specific to gh secret set — any $stringVar | native-exe pipe in Windows PowerShell 5.1 is subject to the same corruption. It also does not reliably show up as an error; the secret write succeeds, so the corruption is silent until something downstream parses the value strictly (URL parsers, JWT parsers, exact-match auth headers, etc).

How to confirm this is the cause

You cannot read back a GH secret’s value (write-only by design). Confirm via a byte-length probe on the source value instead, before ever writing it — this proves whether your extraction value is clean, so you know if the pipe step is the only thing that can be corrupting it:

$expectedLen = $env:VAL.Length   # character count of the clean source string
$tmp = "$env:TEMP\probe.bin"
cmd /c "type `"$srcFile`" > `"$tmp`""   # byte-exact copy via cmd, no PowerShell string pipe
(Get-Item $tmp).Length            # should equal $expectedLen for pure ASCII

If a downstream consumer fails with a parse/format error immediately after a $var | gh secret set write, and the source value passes the probe above, the pipe step is the prime suspect.

Fix — byte-exact write, no PowerShell string pipe

# 1. Decrypt into a variable as usual (never print it)
$env:SOPS_AGE_KEY_FILE = "C:\Users\konar\.age\p24-infra-keys.txt"
$val = (sops -d --input-type dotenv --output-type dotenv <path-to>.env.sops |
    Select-String "^KEY_NAME=").ToString().Split("=",2)[1]
$env:SOPS_AGE_KEY_FILE = ""
 
# 2. Write to a temp file with NO BOM and NO trailing newline
$srcFile = "$env:TEMP\claude\secret_val.txt"
[System.IO.File]::WriteAllText($srcFile, $val, [System.Text.UTF8Encoding]::new($false))
$val = $null
 
# 3. Verify byte length matches expected character count (sanity check)
(Get-Item $srcFile).Length   # compare to the known/expected length
 
# 4. Feed the file into gh via cmd's native `<` redirection — NOT a PowerShell `|` pipe
cmd /c "gh secret set KEY_NAME --repo owner/repo < `"$srcFile`""
 
# 5. Clean up
Remove-Item $srcFile -Force

The key difference from the broken pattern: cmd /c "... < file" redirects raw file bytes directly into the native process’s stdin — it never passes through PowerShell’s object pipeline, so no BOM/CRLF gets injected. Verified byte-exact via cmd /c "type file > file2" producing an identical byte count to the source.

What NOT to do

  • Do not use $env:VAR | gh secret set NAME or $stringVar | gh secret set NAME — corrupts the value with a leading BOM + trailing CRLF.
  • Do not trust gh secret list’s updated timestamp as proof the value is correct — it only confirms a write happened, not that the bytes are clean.
  • Do not assume a receiving tool (e.g. sort.exe) is a safe byte-fidelity test — some Windows console tools reformat their own output (adding BOM/CRLF independently of the input). Use cmd /c "type src > dst" (pure copy, no transform) to isolate pipe-vs-tool corruption.

Where else this pattern appears (audit candidates)

The $env:VAR | some-command pattern is used throughout this ecosystem’s credential-handling guidance (see the global CLAUDE.md “Safe pattern” examples). Most of those examples pipe into some-command --password $env:VAR-style invocations where the variable is passed as an argument, not through the pipeline — that form is unaffected (arguments aren’t subject to this pipeline reformatting, only piped stdin is). The corruption is specific to VALUE | native.exe where native.exe reads from stdin. Any playbook step that pipes a decrypted secret into a native executable’s stdin on Windows should be checked against this pattern and switched to the file-redirection form above.

  • Discovered while fixing P24_DISCORD_INFRA_SCRIPTS_ERRORS_WEBHOOK_URL drift for radieu/et-operational-platform#1412 (2026-08-05) — the first resync attempt via | reproduced the exact same curl exit-3 failure the fix was meant to resolve.
  • sops-windows-crlf.md — a related but distinct Windows text-encoding pitfall (SOPS file writes via Out-File/Set-Content adding CRLF/BOM to files, not secret-manager stdin pipes).

Graph View