Batch API

Run Chat Completions, Responses, and native Anthropic Message Batches asynchronously

Overview

The NanoGPT Batch API runs many independent requests asynchronously at a lower token price than the equivalent synchronous calls. It is a good fit for classification, summarization, evals, synthetic data, document processing, and image analysis where immediate results are not required.

Choose a workflow:

• File-backed API: upload JSONL, create a batch, poll its status, then download output and error files. This OpenAI-compatible workflow is best for large or reusable inputs.
• Inline API: send the requests in the batch creation body and receive the results when polling the completed batch. This is simpler for smaller jobs.
• Anthropic Message Batches: send native Claude requests with the official Anthropic SDK or HTTP API. Requests run directly through Anthropic at batch token rates, with native JSONL results.

The file-backed and inline workflows support rows targeting /v1/chat/completions or /v1/responses. Native Message Batches use a separate /v1/messages/batches endpoint.

All rows in one batch must use the same endpoint and the same model. Batch requests are non-streaming and the only supported completion window is 24h.

Authentication and base URLs

All requests require an API key:

Authorization: Bearer $NANOGPT_API_KEY

Use the dedicated API host for batch operations:

Workflow	Base URL
File-backed	https://api.nano-gpt.com/api/v1
Inline	https://api.nano-gpt.com/api/beta
Anthropic SDK	https://api.nano-gpt.com/api (the SDK appends /v1/messages/batches)

Use api.nano-gpt.com for uploads. The main website host can reject larger multipart requests before they reach the Batch API.

Supported row endpoints

Each row in a file-backed or inline batch must target one of these endpoints:

• /v1/chat/completions
• /v1/responses

Completions, embeddings, image generation, audio, video, transcription, TTS, moderation, and other endpoints cannot be used as batch rows. For native Anthropic request bodies, use Message Batches; /v1/messages is not a supported row endpoint for /api/v1/batches.

Model support

Chat Completions batches

/v1/chat/completions batches support OpenAI, Claude, and Gemini models, and open-source models such as openai/gpt-oss-120b, moonshotai/kimi-k3, z-ai/glm-5.2, and deepseek/deepseek-v4-pro. Always submit the canonical NanoGPT model slug; upstream routing is handled automatically.

Open-source model rows run through NanoGPT’s normal routing as individual requests, started at about one row per second, so they behave like synchronous requests: they cost the same or less, can choose a provider with the request’s provider field, and accept tools and response_format. They do not accept NanoGPT add-ons such as web search, scraping, memory, characters, or BYOK, and every token limit they set must be greater than 0. Rows for OpenAI, Claude, and Gemini models run on the upstream batch API and do not accept tools, functions, response_format, or structured_outputs.

The :thinking suffix is accepted where a thinking variant is configured. Claude thinking aliases are also accepted for compatible models; a numeric thinking budget must be lower than max_tokens.

For the live list, call GET /api/v1/models?detailed=true and select models whose supported_batch_endpoints include /v1/chat/completions.

Model availability changes. Validate a small batch before submitting a large job; an unsupported model returns an unsupported_model error.

Responses batches

/v1/responses batches currently support direct OpenAI models only. The openai/ prefix is accepted. gpt-5.2-pro and gpt-5.4-pro, including dated snapshots, are not available through the upstream Responses Batch API and are rejected.

Responses rows support function and custom tools, structured text output, and remote or data-URL image inputs. Provider-hosted tools and stateful Responses features are not supported in batches.

File-backed API

Endpoints

• POST /files
• GET /files
• GET /files/{file_id}
• GET /files/{file_id}/content
• DELETE /files/{file_id}
• POST /batches
• GET /batches
• GET /batches/{batch_id}
• POST /batches/{batch_id}/cancel

JSONL rules

Each non-empty line must be a JSON object with:

• a unique, non-empty custom_id
• method: "POST"
• url set to /v1/chat/completions or /v1/responses
• a body object containing the model and endpoint-specific input

All rows must use the same endpoint and model. stream: true is rejected.

Chat Completions rows require a non-empty messages array and a positive max_tokens or max_completion_tokens. They support text and compatible image_url content. Image URLs may use HTTP, HTTPS, or base64 data URLs for PNG, JPEG, GIF, and WebP images.

Responses rows require a non-empty input and an integer max_output_tokens of at least 16.

Chat Completions example

{"custom_id":"request-1","method":"POST","url":"/v1/chat/completions","body":{"model":"gpt-4.1-mini","messages":[{"role":"user","content":"Summarize this in one sentence: Batch APIs are useful for offline jobs."}],"max_tokens":64}}
{"custom_id":"request-2","method":"POST","url":"/v1/chat/completions","body":{"model":"gpt-4.1-mini","messages":[{"role":"user","content":"Classify this review as positive or negative: I loved the product."}],"max_tokens":16}}

Responses example

{"custom_id":"response-1","method":"POST","url":"/v1/responses","body":{"model":"gpt-4.1-mini","input":"Extract three keywords from: Batch APIs process independent prompts asynchronously.","max_output_tokens":64}}
{"custom_id":"response-2","method":"POST","url":"/v1/responses","body":{"model":"gpt-4.1-mini","input":"Reply with a JSON object containing a one-sentence summary.","text":{"format":{"type":"json_schema","name":"summary","strict":true,"schema":{"type":"object","properties":{"summary":{"type":"string"}},"required":["summary"],"additionalProperties":false}}},"max_output_tokens":128}}

Upload the file

curl https://api.nano-gpt.com/api/v1/files \
  -H "Authorization: Bearer $NANOGPT_API_KEY" \
  -F purpose=batch \
  -F file=@batch.jsonl

Create the batch

Set endpoint to the same endpoint used by every JSONL row:

curl https://api.nano-gpt.com/api/v1/batches \
  -H "Authorization: Bearer $NANOGPT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input_file_id": "file_abc123",
    "endpoint": "/v1/responses",
    "completion_window": "24h"
  }'

Poll, cancel, and download

# Retrieve one batch
curl https://api.nano-gpt.com/api/v1/batches/batch_abc123 \
  -H "Authorization: Bearer $NANOGPT_API_KEY"

# Best-effort cancellation
curl -X POST https://api.nano-gpt.com/api/v1/batches/batch_abc123/cancel \
  -H "Authorization: Bearer $NANOGPT_API_KEY"

# Download JSONL output
curl https://api.nano-gpt.com/api/v1/files/file_output123/content \
  -H "Authorization: Bearer $NANOGPT_API_KEY"

Batch statuses are validating, in_progress, finalizing, completed, failed, expired, cancelling, and cancelled. A completed batch normally has an output_file_id; row failures may also produce an error_file_id.

List and delete files

Uploaded input files count toward a staged storage allowance of 1 GiB and 100 files per API key until a batch starts from them. An upload that would exceed it returns 429 with batch_file_storage_quota_exceeded and the space in use. Delete input files you no longer need, and output or error files you have downloaded:

# List files, newest first (purpose, limit 1-10000, order asc|desc, after=<last_id>)
curl "https://api.nano-gpt.com/api/v1/files?purpose=batch&limit=100" \
  -H "Authorization: Bearer $NANOGPT_API_KEY"

# Delete a file
curl -X DELETE https://api.nano-gpt.com/api/v1/files/file_abc123 \
  -H "Authorization: Bearer $NANOGPT_API_KEY"

The list follows the OpenAI format (object, data, first_id, last_id, has_more), so client.files.list() and client.files.delete() in the OpenAI SDKs work with these routes. Deletion returns {"id": "file_abc123", "object": "file", "deleted": true} and removes the file’s stored content; batches keep the file ID, but the file can no longer be retrieved, downloaded, or used to create a batch. A file used by a batch that is still running or settling returns 409 with file_in_use; retry after the batch finishes, or cancel it first. Files of inline batches and native Message Batches stay with their batch and also return 409.

Inline API

Inline batches accept up to 10,000 requests, 20 MiB of normalized input, and 250,000 aggregate requested output tokens. Use the file-backed API for larger inputs.

Endpoints

• POST /batches
• GET /batches
• GET /batches/{batch_id}
• POST /batches/{batch_id}/cancel

All inline routes use the https://api.nano-gpt.com/api/beta base URL.

Create an inline batch

curl https://api.nano-gpt.com/api/beta/batches \
  -H "Authorization: Bearer $NANOGPT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "endpoint": "/v1/responses",
    "model": "gpt-4.1-mini",
    "requests": [
      {
        "custom_id": "response-1",
        "body": { "input": "Summarize this text in one sentence." }
      }
    ]
  }'

Creation returns 202 Accepted. Poll GET /api/beta/batches/{batch_id} and read results after completion. Cancel an active job with POST /api/beta/batches/{batch_id}/cancel.

The batch-level model is inherited by every request body. If an inline row omits its endpoint-specific output cap, NanoGPT applies a 4096-token default. File-backed rows must always include the output cap explicitly.

List batches

Use the authenticated list route for the workflow you want to inspect. Each route returns only batches visible to the supplied API key, ordered newest first. Native Message Batches have their own list route and cursor names.

Workflow	Endpoint
Inline	GET https://api.nano-gpt.com/api/beta/batches
All batches (OpenAI-compatible)	GET https://api.nano-gpt.com/api/v1/batches

Both routes support the same filters and bidirectional cursor pagination:

Parameter	Description
limit	Page size from 1 to 100. Defaults to 20.
after	Return the next page of batches older than this batch ID. Pass the current page’s last_id.
before	Return the adjacent page of batches newer than this batch ID, still ordered newest first. Pass the current page’s first_id.
status	Filter by one or more statuses. Repeat the parameter or provide comma-separated values.
created_after	Return batches created strictly after this boundary.
created_before	Return batches created strictly before this boundary.

after and before cannot be combined. Cursors are batch IDs, not arbitrary tokens. Creation boundaries accept Unix timestamps in seconds or timezone-qualified ISO 8601/RFC 3339 timestamps with no more than millisecond precision.

Supported status values are validating, in_progress, finalizing, completed, failed, expired, cancelling, and cancelled.

Filter inline batches

curl "https://api.nano-gpt.com/api/beta/batches?limit=20&status=in_progress,completed" \
  -H "Authorization: Bearer $NANOGPT_API_KEY"

You can also filter by a strict creation-time range:

curl "https://api.nano-gpt.com/api/beta/batches?created_after=2026-09-01T00:00:00Z&created_before=2026-09-02T00:00:00Z" \
  -H "Authorization: Bearer $NANOGPT_API_KEY"

Filter all batches

curl "https://api.nano-gpt.com/api/v1/batches?limit=50&status=failed&created_after=1788192000" \
  -H "Authorization: Bearer $NANOGPT_API_KEY"

Response and pagination

Both endpoints return the standard list envelope:

{
  "object": "list",
  "data": [],
  "first_id": null,
  "last_id": null,
  "has_more": false
}

• data is always ordered newest first.
• first_id and last_id are null for an empty page.
• To fetch older batches, send after=<last_id> with the same filters.
• To return to the adjacent newer page, send before=<first_id> with the same filters.
• has_more indicates that more records exist in the direction being paged.

For example:

# Fetch the next, older page
curl "https://api.nano-gpt.com/api/v1/batches?limit=20&status=completed&after=batch_older_cursor" \
  -H "Authorization: Bearer $NANOGPT_API_KEY"

# Return to the adjacent newer page
curl "https://api.nano-gpt.com/api/v1/batches?limit=20&status=completed&before=batch_newer_cursor" \
  -H "Authorization: Bearer $NANOGPT_API_KEY"

Inline list responses contain metadata only. results is always null, including for completed batches. Retrieve GET /api/beta/batches/{batch_id} to obtain a completed inline batch’s results.

Inline list items include id, object, endpoint, model, completion_window, status, created_at, finalized_at, request_counts, usage, error, and a results field that is always null.

The OpenAI-compatible list includes both inline and file-backed jobs. Its items retain the standard batch shape, including input_file_id, output_file_id, error_file_id, lifecycle timestamps, request_counts, metadata, usage, and errors where applicable.

Listing errors

Both list routes return:

• 401 for missing or invalid authentication.
• 400 when after and before are both present.
• 400 for an unknown or inaccessible cursor. The inline-only route also rejects a cursor that identifies a non-inline batch. A cursor does not need to match the active filters; it only supplies the page boundary.
• 400 for an unsupported status value.
• 400 for an invalid creation timestamp or a reversed creation-time range.

Anthropic Message Batches

Use native Anthropic request shapes for asynchronous Claude jobs, including custom tools, thinking, prompt caching, images, and documents where the selected model supports them. Submit up to 100,000 requests for one supported Claude model per batch. Requests go directly to Anthropic and use batch token pricing, paid from your NanoGPT balance.

The official Anthropic SDK works with a NanoGPT API key and baseURL: 'https://api.nano-gpt.com/api' (Python: base_url). It appends /v1/messages/batches. The shorter /v1/messages/batches alias also works with https://api.nano-gpt.com as the SDK base URL.

Native endpoints

Method	URL path	Purpose
POST	/api/v1/messages/batches	Create a batch
GET	/api/v1/messages/batches	List native Message Batches
GET	/api/v1/messages/batches/{id}	Retrieve status and request counts
POST	/api/v1/messages/batches/{id}/cancel	Request cancellation
GET	/api/v1/messages/batches/{id}/results	Download native result JSONL
DELETE	/api/v1/messages/batches/{id}	Delete an ended batch from NanoGPT’s native API

All routes also accept the /v1/messages/batches prefix. Authenticate with x-api-key or Authorization: Bearer using your NanoGPT API key.

Create a native batch

Send a requests array containing custom_id and native Messages params. No file upload is required.

curl https://api.nano-gpt.com/api/v1/messages/batches \
  -H "x-api-key: $NANOGPT_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: example-anthropic-batch-001" \
  -d '{
    "requests": [{
      "custom_id": "summary-1",
      "params": {
        "model": "anthropic/claude-sonnet-5.5",
        "max_tokens": 1024,
        "messages": [{ "role": "user", "content": "Explain when batch processing is useful." }]
      }
    }]
  }'

Or use @anthropic-ai/sdk:

import Anthropic from '@anthropic-ai/sdk';

const client = new Anthropic({
  apiKey: process.env.NANOGPT_API_KEY,
  baseURL: 'https://api.nano-gpt.com/api',
});

const batch = await client.messages.batches.create({
  requests: [{
    custom_id: 'summary-1',
    params: {
      model: 'anthropic/claude-sonnet-5.5',
      max_tokens: 1024,
      messages: [{ role: 'user', content: 'Explain when batch processing is useful.' }],
    },
  }],
}, { headers: { 'Idempotency-Key': 'example-anthropic-batch-001' } });

// Retrieve periodically until processing_status is "ended".
const current = await client.messages.batches.retrieve(batch.id);
if (current.processing_status === 'ended' && current.results_url) {
  for await (const row of await client.messages.batches.results(batch.id)) {
    console.log(row.custom_id, row.result);
  }
}

Choose a new Idempotency-Key for each distinct batch. Reuse the same key and body when retrying a submission; a different payload with the same key returns 409. A stored creation failure can also be replayed, so use a new key for a new submission after correcting that failure. The header is optional, but recommended for retries.

Native request limits and compatibility

• Send 1–100,000 requests with unique, non-empty custom_id values.
• Use the same supported Claude model for every request. Thinking controls may vary by request. Support for a model on synchronous /v1/messages does not imply native batch support.
• Haiku 5.5 accepts anthropic/claude-haiku-5.5, claude-haiku-5-5, and anthropic/claude-haiku-latest. Use thinking: { "type": "adaptive" } with output_config: { "effort": "medium" } to set its default thinking behavior explicitly; fixed budget_tokens controls are unsupported by Haiku 5.5.
• Haiku 5.5 also accepts thinking: { "type": "disabled" } at low, medium, or high effort. Disabled thinking with xhigh or max is rejected.
• Each params requires model, messages, and a positive integer max_tokens within the model’s output limit. NanoGPT’s maximum accepted cap is 1,000,000; model limits can be lower. The first message must have the user role; use the top-level system field for a system prompt.
• The request body and stored normalized input must each fit within 200 MiB. Normalization can make the stored input larger than the original body.
• The completion window is 24 hours. Streaming generation is unsupported; result downloads use JSONL.
• Native custom tools and tool-result history, thinking, output_config, and prompt-cache controls are supported where the selected model supports them. Use native thinking controls, not OpenAI reasoning or reasoning_effort fields.
• Images accept URL or base64 sources; documents accept URL, base64, or plaintext sources. Provider-managed file references, containers, MCP servers, hosted tools, and video blocks are rejected.
• URL or base64 documents reserve enough balance for a full context window because their token count is unknown before processing. Final billing still uses actual token usage.
• Caller anthropic-beta flags are forwarded alongside any flags required by the request. A beta header does not enable fields rejected by NanoGPT’s batch validation.
• Keys or teams requiring Zero Data Retention cannot create batches because batch processing retains input.

Native Message Batches use the official Anthropic SDK. NanoGPT’s general-purpose SDKs do not expose these batch methods.

Native status, results, cancellation, and deletion

Creation returns a message_batch object. Poll retrieval until processing_status is ended; the other states are in_progress and canceling. An ended batch can contain unsuccessful requests, so inspect request_counts and each result:

• request_counts contains processing, succeeded, errored, canceled, and expired.
• results_url is available when results can be downloaded. Results are native JSONL rows with custom_id and result; match by custom_id and inspect result.type, rather than assuming input order or success.
• Results requested before they are available return 409. Download available results before provider archival; an ended status alone does not guarantee results remain available.
• Cancellation is best-effort. Requests that complete can still incur usage charges.
• Delete only after processing ends. Deletion hides the batch from NanoGPT’s native API; it does not reverse billing or promise deletion of Anthropic’s retained data.

List native Message Batches

GET /api/v1/messages/batches returns native batches visible to the supplied API key, newest first, with data, has_more, first_id, and last_id.

Use limit from 1 to 100 (default 20), after_id=<last_id> for older batches, or before_id=<first_id> for newer batches. Do not combine both cursor parameters. Use the native list for native Message Batches; the /api/v1/batches list uses the OpenAI-compatible representation.

Responses Batch restrictions

Responses Batch is stateless and executes directly through the upstream batch service. NanoGPT forces store: false and rejects:

• previous_response_id, conversation, and background: true
• reusable prompt references
• input_file, item_reference, video inputs, and input_image.file_id
• provider-hosted tools; function and custom tools remain supported
• NanoGPT-only features such as Advisor, memory, scraping, retention overrides, provider or BYOK controls, caching controls, and billing overrides

Remote HTTP(S) image URLs and image data URLs are accepted. Structured output through the Responses text.format field is supported.

Billing

Batch jobs use NanoGPT account balance and do not use subscription included tokens. At creation, NanoGPT checks the balance against a conservative maximum-liability estimate based on the input and output caps. Completed usage is charged once after the batch reaches a terminal state. If there is no billable usage, no usage charge is created.

• OpenAI, Claude, and Gemini models, and some fine-tuned models: token usage is priced 50% below the equivalent synchronous request. Non-token charges, where supported, keep their normal rate.
• Other open-source models: each row is charged exactly what the same synchronous request costs, including the automatic routing price, your subscriber discount, and the price of any provider you select. Rows are priced in USD, and a batch never costs more than the amount reserved when it was created.
• Open-source models with a batch price: some models, such as moonshotai/kimi-k3, have a lower batch price for rows that use automatic routing. An alias such as moonshotai/kimi-latest uses the batch price of the model it points to. A row that sets provider pays that provider’s normal price instead. A row never costs more than the same synchronous request.

Each model page shows its Batch API price. Use the live pricing page or pricing API as the source of truth.

Common errors

• Unsupported endpoint: file-backed and inline rows must use /v1/chat/completions or /v1/responses, consistently across all rows. Native Claude requests use /v1/messages/batches.
• Missing output cap: add max_tokens or max_completion_tokens for Chat Completions, or max_output_tokens >= 16 for Responses.
• Mixed model: every row must use the same model.
• Streaming unsupported: remove stream: true.
• Unsupported Responses model: choose a direct OpenAI model supported by the upstream Batch API.
• Unsupported Responses field: remove stateful features, provider-hosted tools, file references, or NanoGPT-only extensions.