p24-infra Docs

gitlab-commit-api-windows-powershell

5 min read

Playbook: GitLab Commits/Variables API from Windows PowerShell — Corruption Gotchas

Applies to: any p24-infra session on the Windows dev machine pushing file content or CI variables to a GitLab repository (e.g. pinbox24/*) via the REST API using PowerShell + curl.exe.

Discovered: 2026-08-01, while remediating p24-infra#2697 (hardcoded credentials in pinbox24/p24-ms-mailgun’s docker-deploy-prod.sh). Two independent Windows-only bugs corrupted an API push that looked successful (HTTP 201, no error) — both are silent, so always verify by re-fetching, not by trusting the response code alone.

Bug 1 — Get-Content -Raw + ConvertTo-Json explodes file content into a huge PSObject dump

Symptom

A file committed via POST /projects/:id/repository/commits (or any GitLab API call whose JSON body embeds file content built from Get-Content -Raw) ends up on the branch as a multi-hundred-KB blob that looks like a serialized PowerShell object ({:value=>"...", :PSPath=>"...", :PSDrive=>[{:Name=>"E", :Root=>"E:\\", ...}], :ReadCount=>1}) instead of the plain text you intended — even though the API call returned 201 with no error.

A quick substring check (e.g. $content -match 'some-expected-string') can give a false positive, because the expected substring is still present verbatim somewhere inside the giant dump. Only a length/byte-exact check catches it.

Root cause

Get-Content -Raw returns a System.String, but the FileSystem provider attaches hidden ETS (Extended Type System) note properties to it — PSPath, PSParentPath, PSChildName, PSDrive, PSProvider. When that string is nested inside a hashtable/array structure and serialized with ConvertTo-Json -Depth N (N greater than ~2), PowerShell 5.1 can recurse into those hidden ETS members — including the full local PSDrive list (all drive letters, descriptions, credentials placeholder) — instead of emitting a plain JSON string. Depth budget consumed by the outer structure determines whether you get a bounded-but-huge blob (hundreds of KB, nested inside an array) or an unbounded explosion (100M+ chars, if the string is a top-level hashtable value).

Plain strings created via .Substring(), string interpolation, or [System.IO.File]::ReadAllText() do not carry this ETS wrapper and serialize normally.

Fix

Always read file content that will be JSON-serialized for an API payload with:

$content = [System.IO.File]::ReadAllText($path)

never Get-Content -Raw for this purpose (Get-Content -Raw is fine for content you only read/inspect locally, or that gets re-flattened through a plain-string operation like .Substring() or "$(...)" before being embedded in JSON).

Verification (mandatory after every commit-api push)

Before considering a GitLab commit-api push done, re-fetch the raw file from the branch and check:

$verify = Get-Content $verifyFile -Raw   # local read for inspection only, not re-serialized
$verify.Length -eq $expectedLength                    # byte-length sanity check
$verify -notlike "*PSDrive*"                           # corruption marker
$verify -eq $intendedContent                           # exact match, not just substring presence

A HTTP 201 alone is not sufficient evidence the push worked correctly.

Bug 2 — curl.exe mangles secret values containing shell-special characters

Symptom

Setting a GitLab CI/CD variable (POST /projects/:id/variables) via multiple separate --data-urlencode / --data arguments to curl.exe from PowerShell can silently concatenate a later --data flag’s literal text onto the end of an earlier argument’s value — e.g. a secret value ending up stored as <real-secret-value> --data masked=true --data protected=true. The API call still returns 201; nothing looks wrong unless you check the resulting variable’s value length against the source.

Trigger characters seen in practice: & ! # $ % ^ * ( ) :. Values containing these (common in tokens and generated passwords) are at risk.

Fix

Build the entire application/x-www-form-urlencoded request body as one string in PowerShell, using [uri]::EscapeDataString() on every field value, write it to a temp file, and send it with --data-binary "@file" — never pass a raw secret as a separate curl.exe command-line argument:

$body = "key=" + [uri]::EscapeDataString($keyName) + "&value=" + [uri]::EscapeDataString($secretValue) + "&masked=$masked&protected=true"
[System.IO.File]::WriteAllText($bodyFile, $body, [System.Text.UTF8Encoding]::new($false))
curl.exe -s -o $respFile -w "%{http_code}" -H "PRIVATE-TOKEN: $env:GITLAB_ADMIN_PAT" -H "Content-Type: application/x-www-form-urlencoded" -X POST "$uri" --data-binary "@$bodyFile"

Verification

After creating/updating a variable, compare length (never value) against the source:

$obj.value.Length -eq $secretValue.Length      # or a hash comparison if you want stronger assurance
$obj.value -notlike "*--data*"                 # corruption marker for this specific bug

If corrupted: DELETE /projects/:id/variables/:key then recreate with the fixed pattern above. A DELETE immediately followed by POST for the same key can transiently 400 (“value is invalid”) on GitLab’s side — retry once after the delete if this happens; it is not the same bug.

Bug 3 (not Windows-specific, but hit alongside the above) — GitLab variable masking rejects punctuation-heavy values

masked=true requires the value to match GitLab’s restricted charset (roughly base64 alphabet plus a handful of extra characters) and forbids whitespace, ?, and , among others. A MongoDB connection string with multiple hosts (,) and query parameters (?...&...) — or a generated token containing punctuation like !@#$%^&*() — will 400 with {"message":{"value":["is invalid"]}} when masked=true.

Fix: for such values, use protected=true + masked=false instead of failing the whole operation. Document this explicitly in the commit/MR (masking is a defense-in-depth nicety, not a substitute for protected=true restricting the variable to protected branches/environments). Check before setting: $value -match '[^A-Za-z0-9@:.~^!$%&()+=/{}\[\]_''\-]' (i.e. contains a comma, question mark, whitespace, or other char outside the safe set) → use masked=false.

General reminder — env vars do not persist across tool calls

PowerShell environment variables set in one tool invocation are gone in the next (each call is a fresh process on this harness). Any workflow that extracts a SOPS secret into $env:X and then uses it against an external API must do both the extraction and the use inside the same PowerShell call/script — see project_supabase_token_false_expiry_2026_07_11 in session memory for the same lesson learned earlier with Supabase tokens.

Graph View

Backlinks