anthropic-batch-api

Anthropic Batch API — p24-infra Playbook

What it is

The Anthropic Batch API (POST /v1/messages/batches) processes requests asynchronously at 50% of synchronous API prices. It is designed for non-real-time workloads where latency is not critical.

• Up to 100,000 requests or 256 MB per batch
• Most batches complete within 1 hour; maximum processing time is 24 hours
• Results are available for 29 days after creation
• Supports all Messages API features: vision, tool use, caching, extended thinking
• Results are unordered — always match results to requests by custom_id

CRITICAL: Who can use it (p24-infra)

Only consumers that authenticate via ANTHROPIC_API_KEY can use the Batch API.

The Batch API is a direct HTTP API. It does NOT work with:

• claude --dangerously-skip-permissions -p CLI workers (OAuth subscription)
• Any worker running on vps-i1 (AI-Dev-IO1) or bms-4 (AI-Dev-BMS4-1)
• The dispatch pipeline (dev_r_worker_queue)

In p24-infra, the only current ANTHROPIC_API_KEY consumer is audit-engine. All queue workers use OAuth subscription (Claude Max) — Batch API does not apply to them.

Consumer	Auth method	Batch API usable?
audit-engine (vps-i1, not yet deployed)	ANTHROPIC_API_KEY direct	YES
dev_r_worker_queue workers (bms-4, vps-i1)	OAuth subscription (claude -p)	NO
review-pr skill workers	OAuth subscription	NO
credential-exporter health-check pings	API key, read-only	Pointless (one call)

When to use in audit-engine

Good candidates — non-real-time, batching saves real money:

Workbook type	Schedule	Rationale
EU AI Act compliance checks	Weekly (Monday 08:00 UTC)	Up to 24h acceptable, multiple systems in one batch
infra_docs_check across all services	Daily	Scan multiple dev_r_services rows in one batch
Workbook design for status=new actions	On-demand	Async design, write result to audit.workbooks
Report generation (PDF workbooks)	Scheduled	Long-running, no need for immediate result

Do NOT use Batch API for:

• infra_alert actions (need immediate response)
• Interactive queries (anything where the user is waiting)

Cost comparison (current models, 2026-06)

Model	Sync input	Sync output	Batch input (−50%)	Batch output (−50%)
Sonnet 4.6	$3.00/1M	$15.00/1M	$1.50/1M	$7.50/1M
Opus 4.8	$5.00/1M	$25.00/1M	$2.50/1M	$12.50/1M
Haiku 4.5	$1.00/1M	$5.00/1M	$0.50/1M	$2.50/1M

For audit-engine workbook design (typically ~10–30k tokens/workbook, Opus), batch saves ~$0.05–0.15 per workbook. With hundreds of monthly runs the saving is meaningful.

Python implementation pattern

The audit-engine uses the anthropic Python SDK directly.

Create a batch

import anthropic
 
client = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY from env
 
batch = client.messages.batches.create(
    requests=[
        {
            "custom_id": f"action-{action_id}-{run_date}",
            "params": {
                "model": "claude-opus-4-8",
                "max_tokens": 8096,
                "thinking": {"type": "adaptive"},
                "messages": [
                    {
                        "role": "user",
                        "content": workbook_prompt
                    }
                ]
            }
        }
        for action_id, workbook_prompt in pending_actions
    ]
)
 
print(f"Batch created: {batch.id}, requests: {batch.request_counts.processing}")
# Store batch.id in audit.runs for polling

Poll until complete

Poll on a schedule — do NOT busy-wait. Batches typically finish in <1 hour.

import time
 
def wait_for_batch(client: anthropic.Anthropic, batch_id: str, poll_interval: int = 60) -> None:
    while True:
        batch = client.messages.batches.retrieve(batch_id)
        counts = batch.request_counts
        print(
            f"Batch {batch_id}: processing={counts.processing} "
            f"succeeded={counts.succeeded} errored={counts.errored} expired={counts.expired}"
        )
        if batch.processing_status == "ended":
            break
        time.sleep(poll_interval)

Retrieve and process results

Results are an iterator (streaming download) — order is NOT guaranteed. Match by custom_id.

def process_batch_results(client: anthropic.Anthropic, batch_id: str) -> dict:
    results = {}
    for result in client.messages.batches.results(batch_id):
        if result.result.type == "succeeded":
            # Extract text from the first text block
            text_blocks = [b for b in result.result.message.content if b.type == "text"]
            results[result.custom_id] = text_blocks[0].text if text_blocks else ""
        elif result.result.type == "errored":
            print(f"ERROR custom_id={result.custom_id}: {result.result.error}")
            results[result.custom_id] = None
        elif result.result.type == "expired":
            print(f"EXPIRED custom_id={result.custom_id}: request timed out after 24h")
            results[result.custom_id] = None
    return results

Cancel a batch (if needed)

client.messages.batches.cancel(batch_id)
# Already-completed requests within the batch are still billed

audit-engine integration pattern

The scheduler in audit-engine/scheduler.py runs workbook design on status=new actions. When implementing batch support, the pattern is:

1. APScheduler fires a “batch design” job (separate from the per-action cron jobs)
2. Collect all audit.actions rows where status='new'
3. Submit one batch containing a design request per action
4. Store batch_id in a new audit.batch_jobs table (or in audit.runs.metadata)
5. A second APScheduler job polls every 30 minutes and processes completed results
6. On result: write WorkbookSpec JSON to audit.workbooks, set audit.actions.status='active'

This avoids holding a connection open for 24 hours and handles APScheduler restarts gracefully.

Key API gotchas

Gotcha	Detail
Results are unordered	Always use custom_id to match requests to results
expired requests	24-hour hard limit — requests not processed by then return expired
Partial billing	Each succeeded request is billed individually; errored/expired are NOT billed
thinking blocks	Supported, same as sync API. budget_tokens is deprecated on Opus 4.8+ — use {type: "adaptive"}
Extended context	Same 1M context window as sync API
Prompt caching	Supported in batch requests — cache breakpoints on input tokens still apply
Rate limits	Batch requests count against requests_per_minute limits (at creation time, not result time)

References

• Anthropic Batch API docs: https://docs.anthropic.com/en/api/creating-message-batches
• Python SDK: client.messages.batches.* (anthropic>=0.40.0)
• audit-engine source: audit-engine/ai.py, audit-engine/scheduler.py
• ANTHROPIC_API_KEY location: secrets/monitoring.env.sops (ANTHROPIC_API_KEY)

See also

• audit-engine.md — full audit-engine specification
• anthropic-api-key-rotation.md — key rotation procedure