Batch API

Run high-volume chat completions and Responses API requests asynchronously

Overview

The NanoGPT Batch API runs many independent requests asynchronously at a lower token price than the equivalent synchronous calls. It is a good fit for classification, summarization, evals, synthetic data, document processing, and image analysis where immediate results are not required.

You can submit a batch in either of two ways:

• File-backed API: upload JSONL, create a batch, poll its status, then download output and error files. This OpenAI-compatible workflow is best for large or reusable inputs.
• Inline API: send the requests in the batch creation body and receive the results when polling the completed batch. This is simpler for smaller jobs.

Both workflows support rows targeting /v1/chat/completions or /v1/responses.

All rows in one batch must use the same endpoint and the same model. Batch requests are non-streaming and the only supported completion window is 24h.

Authentication and base URLs

All requests require an API key:

Authorization: Bearer $NANOGPT_API_KEY

Use the dedicated API host for batch operations:

Workflow	Base URL
File-backed	https://api.nano-gpt.com/api/v1
Inline	https://api.nano-gpt.com/api/beta

Use api.nano-gpt.com for uploads. The main website host can reject larger multipart requests before they reach the Batch API.

Supported row endpoints

Each request in a batch must target one of these endpoints:

• /v1/chat/completions
• /v1/responses

Completions, embeddings, image generation, audio, video, transcription, TTS, moderation, and other endpoints cannot be used as batch rows.

Model support

Chat Completions batches

/v1/chat/completions batches support selected direct OpenAI, Claude, Gemini, managed, and Fireworks Batch API models. Current managed examples include MiniMax M3, GLM 5.1 and 5.2, DeepSeek V4 Pro, and Kimi K2.7 Code.

Supported Fireworks Batch API model IDs include:

• accounts/fireworks/models/deepseek-v4-flash
• accounts/fireworks/models/deepseek-v4-pro
• accounts/fireworks/models/glm-5p2
• accounts/fireworks/models/gpt-oss-120b
• accounts/fireworks/models/gpt-oss-20b
• accounts/fireworks/models/inkling
• accounts/fireworks/models/kimi-k2p6
• accounts/fireworks/models/kimi-k2p7-code
• accounts/fireworks/models/kimi-k3
• accounts/fireworks/models/minimax-m2p7
• accounts/fireworks/models/minimax-m3
• accounts/fireworks/models/muse-glimmer-30b
• accounts/fireworks/models/nemotron-lightning-3p5-30b-a3b
• accounts/fireworks/models/nemotron-3-ultra-nvfp4
• accounts/fireworks/models/qwen3p6-plus
• accounts/fireworks/models/qwen3p7-plus

The :thinking suffix is accepted for Fireworks model IDs where a thinking variant is configured. Claude thinking aliases are also accepted for compatible models; a numeric thinking budget must be lower than max_tokens.

Model availability changes. Validate a small batch before submitting a large job; an unsupported model returns an unsupported_model error.

Responses batches

/v1/responses batches currently support direct OpenAI models only. The openai/ prefix is accepted. gpt-5.2-pro and gpt-5.4-pro, including dated snapshots, are not available through the upstream Responses Batch API and are rejected.

Responses rows support function and custom tools, structured text output, and remote or data-URL image inputs. Provider-hosted tools and stateful Responses features are not supported in batches.

File-backed API

Endpoints

• POST /files
• GET /files/{file_id}
• GET /files/{file_id}/content
• POST /batches
• GET /batches/{batch_id}
• GET /batches
• POST /batches/{batch_id}/cancel

JSONL rules

Each non-empty line must be a JSON object with:

• a unique, non-empty custom_id
• method: "POST"
• url set to /v1/chat/completions or /v1/responses
• a body object containing the model and endpoint-specific input

All rows must use the same endpoint and model. stream: true is rejected.

Chat Completions rows require a non-empty messages array and a positive max_tokens or max_completion_tokens. They support text and compatible image_url content. Image URLs may use HTTP, HTTPS, or base64 data URLs for PNG, JPEG, GIF, and WebP images.

Responses rows require a non-empty input and an integer max_output_tokens of at least 16.

Chat Completions example

{"custom_id":"request-1","method":"POST","url":"/v1/chat/completions","body":{"model":"gpt-4.1-mini","messages":[{"role":"user","content":"Summarize this in one sentence: Batch APIs are useful for offline jobs."}],"max_tokens":64}}
{"custom_id":"request-2","method":"POST","url":"/v1/chat/completions","body":{"model":"gpt-4.1-mini","messages":[{"role":"user","content":"Classify this review as positive or negative: I loved the product."}],"max_tokens":16}}

Responses example

{"custom_id":"response-1","method":"POST","url":"/v1/responses","body":{"model":"gpt-4.1-mini","input":"Extract three keywords from: Batch APIs process independent prompts asynchronously.","max_output_tokens":64}}
{"custom_id":"response-2","method":"POST","url":"/v1/responses","body":{"model":"gpt-4.1-mini","input":"Reply with a JSON object containing a one-sentence summary.","text":{"format":{"type":"json_schema","name":"summary","strict":true,"schema":{"type":"object","properties":{"summary":{"type":"string"}},"required":["summary"],"additionalProperties":false}}},"max_output_tokens":128}}

Upload the file

curl https://api.nano-gpt.com/api/v1/files \
  -H "Authorization: Bearer $NANOGPT_API_KEY" \
  -F purpose=batch \
  -F file=@batch.jsonl

Create the batch

Set endpoint to the same endpoint used by every JSONL row:

curl https://api.nano-gpt.com/api/v1/batches \
  -H "Authorization: Bearer $NANOGPT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input_file_id": "file_abc123",
    "endpoint": "/v1/responses",
    "completion_window": "24h"
  }'

Poll, list, cancel, and download

# Retrieve one batch
curl https://api.nano-gpt.com/api/v1/batches/batch_abc123 \
  -H "Authorization: Bearer $NANOGPT_API_KEY"

# List batches; optional query parameters are limit and after
curl https://api.nano-gpt.com/api/v1/batches \
  -H "Authorization: Bearer $NANOGPT_API_KEY"

# Best-effort cancellation
curl -X POST https://api.nano-gpt.com/api/v1/batches/batch_abc123/cancel \
  -H "Authorization: Bearer $NANOGPT_API_KEY"

# Download JSONL output
curl https://api.nano-gpt.com/api/v1/files/file_output123/content \
  -H "Authorization: Bearer $NANOGPT_API_KEY"

Batch statuses are validating, in_progress, finalizing, completed, failed, expired, cancelling, and cancelled. A completed batch normally has an output_file_id; row failures may also produce an error_file_id.

Inline API

Inline batches accept up to 10,000 requests, 20 MiB of normalized input, and 250,000 aggregate requested output tokens. Use the file-backed API for larger inputs.

curl https://api.nano-gpt.com/api/beta/batches \
  -H "Authorization: Bearer $NANOGPT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "endpoint": "/v1/responses",
    "model": "gpt-4.1-mini",
    "requests": [
      {
        "custom_id": "response-1",
        "body": { "input": "Summarize this text in one sentence." }
      }
    ]
  }'

Creation returns 202 Accepted. Poll GET /api/beta/batches/{batch_id} and read results after completion. Cancel an active job with POST /api/beta/batches/{batch_id}/cancel.

The batch-level model is inherited by every request body. If an inline row omits its endpoint-specific output cap, NanoGPT applies a 4096-token default. File-backed rows must always include the output cap explicitly.

Responses Batch restrictions

Responses Batch is stateless and executes directly through the upstream batch service. NanoGPT forces store: false and rejects:

• previous_response_id, conversation, and background: true
• reusable prompt references
• input_file, item_reference, video inputs, and input_image.file_id
• provider-hosted tools; function and custom tools remain supported
• NanoGPT-only features such as Advisor, memory, scraping, retention overrides, provider or BYOK controls, caching controls, and billing overrides

Remote HTTP(S) image URLs and image data URLs are accepted. Structured output through the Responses text.format field is supported.

Billing

Batch jobs use NanoGPT account balance and do not use subscription included tokens. At creation, NanoGPT checks the balance against a conservative maximum-liability estimate based on the input and output caps. Completed usage is charged once after the batch reaches a terminal state. If there is no billable usage, no usage charge is created.

Supported batch token usage is priced 50% below the equivalent synchronous request. Non-token charges, where supported, keep their normal rate. Use the live pricing page or pricing API as the source of truth.

Common errors

• Unsupported endpoint: use /v1/chat/completions or /v1/responses, consistently across all rows.
• Missing output cap: add max_tokens or max_completion_tokens for Chat Completions, or max_output_tokens >= 16 for Responses.
• Mixed model: every row must use the same model.
• Streaming unsupported: remove stream: true.
• Unsupported Responses model: choose a direct OpenAI model supported by the upstream Batch API.
• Unsupported Responses field: remove stateful features, provider-hosted tools, file references, or NanoGPT-only extensions.