> ## Documentation Index
> Fetch the complete documentation index at: https://docs.nano-gpt.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Batch API

> Run high-volume chat completions and Responses API requests asynchronously

## Overview

The NanoGPT Batch API runs many independent requests asynchronously at a lower token price than the equivalent synchronous calls. It is a good fit for classification, summarization, evals, synthetic data, document processing, and image analysis where immediate results are not required.

You can submit a batch in either of two ways:

* **File-backed API:** upload JSONL, create a batch, poll its status, then download output and error files. This OpenAI-compatible workflow is best for large or reusable inputs.
* **Inline API:** send the requests in the batch creation body and receive the results when polling the completed batch. This is simpler for smaller jobs.

Both workflows support rows targeting `/v1/chat/completions` or `/v1/responses`.

<Note>
  All rows in one batch must use the same endpoint and the same model. Batch requests are non-streaming and the only supported completion window is `24h`.
</Note>

## Authentication and base URLs

All requests require an API key:

```http theme={null}
Authorization: Bearer $NANOGPT_API_KEY
```

Use the dedicated API host for batch operations:

| Workflow    | Base URL                            |
| ----------- | ----------------------------------- |
| File-backed | `https://api.nano-gpt.com/api/v1`   |
| Inline      | `https://api.nano-gpt.com/api/beta` |

Use `api.nano-gpt.com` for uploads. The main website host can reject larger multipart requests before they reach the Batch API.

## Supported row endpoints

Each request in a batch must target one of these endpoints:

* `/v1/chat/completions`
* `/v1/responses`

Completions, embeddings, image generation, audio, video, transcription, TTS, moderation, and other endpoints cannot be used as batch rows.

## Model support

### Chat Completions batches

`/v1/chat/completions` batches support selected direct OpenAI, Claude, Gemini, managed, and Fireworks Batch API models. Current managed examples include MiniMax M3, GLM 5.1 and 5.2, DeepSeek V4 Pro, and Kimi K2.7 Code.

Supported Fireworks Batch API model IDs include:

* `accounts/fireworks/models/deepseek-v4-flash`
* `accounts/fireworks/models/deepseek-v4-pro`
* `accounts/fireworks/models/glm-5p2`
* `accounts/fireworks/models/gpt-oss-120b`
* `accounts/fireworks/models/gpt-oss-20b`
* `accounts/fireworks/models/inkling`
* `accounts/fireworks/models/kimi-k2p6`
* `accounts/fireworks/models/kimi-k2p7-code`
* `accounts/fireworks/models/kimi-k3`
* `accounts/fireworks/models/minimax-m2p7`
* `accounts/fireworks/models/minimax-m3`
* `accounts/fireworks/models/muse-glimmer-30b`
* `accounts/fireworks/models/nemotron-lightning-3p5-30b-a3b`
* `accounts/fireworks/models/nemotron-3-ultra-nvfp4`
* `accounts/fireworks/models/qwen3p6-plus`
* `accounts/fireworks/models/qwen3p7-plus`

The `:thinking` suffix is accepted for Fireworks model IDs where a thinking variant is configured. Claude thinking aliases are also accepted for compatible models; a numeric thinking budget must be lower than `max_tokens`.

Model availability changes. Validate a small batch before submitting a large job; an unsupported model returns an `unsupported_model` error.

### Responses batches

`/v1/responses` batches currently support direct OpenAI models only. The `openai/` prefix is accepted. `gpt-5.2-pro` and `gpt-5.4-pro`, including dated snapshots, are not available through the upstream Responses Batch API and are rejected.

Responses rows support function and custom tools, structured text output, and remote or data-URL image inputs. Provider-hosted tools and stateful Responses features are not supported in batches.

## File-backed API

### Endpoints

* `POST /files`
* `GET /files/{file_id}`
* `GET /files/{file_id}/content`
* `POST /batches`
* `GET /batches/{batch_id}`
* `GET /batches`
* `POST /batches/{batch_id}/cancel`

### JSONL rules

Each non-empty line must be a JSON object with:

* a unique, non-empty `custom_id`
* `method: "POST"`
* `url` set to `/v1/chat/completions` or `/v1/responses`
* a `body` object containing the model and endpoint-specific input

All rows must use the same endpoint and model. `stream: true` is rejected.

Chat Completions rows require a non-empty `messages` array and a positive `max_tokens` or `max_completion_tokens`. They support text and compatible `image_url` content. Image URLs may use HTTP, HTTPS, or base64 data URLs for PNG, JPEG, GIF, and WebP images.

Responses rows require a non-empty `input` and an integer `max_output_tokens` of at least `16`.

### Chat Completions example

```jsonl theme={null}
{"custom_id":"request-1","method":"POST","url":"/v1/chat/completions","body":{"model":"gpt-4.1-mini","messages":[{"role":"user","content":"Summarize this in one sentence: Batch APIs are useful for offline jobs."}],"max_tokens":64}}
{"custom_id":"request-2","method":"POST","url":"/v1/chat/completions","body":{"model":"gpt-4.1-mini","messages":[{"role":"user","content":"Classify this review as positive or negative: I loved the product."}],"max_tokens":16}}
```

### Responses example

```jsonl theme={null}
{"custom_id":"response-1","method":"POST","url":"/v1/responses","body":{"model":"gpt-4.1-mini","input":"Extract three keywords from: Batch APIs process independent prompts asynchronously.","max_output_tokens":64}}
{"custom_id":"response-2","method":"POST","url":"/v1/responses","body":{"model":"gpt-4.1-mini","input":"Reply with a JSON object containing a one-sentence summary.","text":{"format":{"type":"json_schema","name":"summary","strict":true,"schema":{"type":"object","properties":{"summary":{"type":"string"}},"required":["summary"],"additionalProperties":false}}},"max_output_tokens":128}}
```

### Upload the file

```bash theme={null}
curl https://api.nano-gpt.com/api/v1/files \
  -H "Authorization: Bearer $NANOGPT_API_KEY" \
  -F purpose=batch \
  -F file=@batch.jsonl
```

### Create the batch

Set `endpoint` to the same endpoint used by every JSONL row:

```bash theme={null}
curl https://api.nano-gpt.com/api/v1/batches \
  -H "Authorization: Bearer $NANOGPT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input_file_id": "file_abc123",
    "endpoint": "/v1/responses",
    "completion_window": "24h"
  }'
```

### Poll, list, cancel, and download

```bash theme={null}
# Retrieve one batch
curl https://api.nano-gpt.com/api/v1/batches/batch_abc123 \
  -H "Authorization: Bearer $NANOGPT_API_KEY"

# List batches; optional query parameters are limit and after
curl https://api.nano-gpt.com/api/v1/batches \
  -H "Authorization: Bearer $NANOGPT_API_KEY"

# Best-effort cancellation
curl -X POST https://api.nano-gpt.com/api/v1/batches/batch_abc123/cancel \
  -H "Authorization: Bearer $NANOGPT_API_KEY"

# Download JSONL output
curl https://api.nano-gpt.com/api/v1/files/file_output123/content \
  -H "Authorization: Bearer $NANOGPT_API_KEY"
```

Batch statuses are `validating`, `in_progress`, `finalizing`, `completed`, `failed`, `expired`, `cancelling`, and `cancelled`. A completed batch normally has an `output_file_id`; row failures may also produce an `error_file_id`.

## Inline API

Inline batches accept up to 10,000 requests, 20 MiB of normalized input, and 250,000 aggregate requested output tokens. Use the file-backed API for larger inputs.

```bash theme={null}
curl https://api.nano-gpt.com/api/beta/batches \
  -H "Authorization: Bearer $NANOGPT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "endpoint": "/v1/responses",
    "model": "gpt-4.1-mini",
    "requests": [
      {
        "custom_id": "response-1",
        "body": { "input": "Summarize this text in one sentence." }
      }
    ]
  }'
```

Creation returns `202 Accepted`. Poll `GET /api/beta/batches/{batch_id}` and read `results` after completion. Cancel an active job with `POST /api/beta/batches/{batch_id}/cancel`.

The batch-level model is inherited by every request body. If an inline row omits its endpoint-specific output cap, NanoGPT applies a 4096-token default. File-backed rows must always include the output cap explicitly.

## Responses Batch restrictions

Responses Batch is stateless and executes directly through the upstream batch service. NanoGPT forces `store: false` and rejects:

* `previous_response_id`, `conversation`, and `background: true`
* reusable `prompt` references
* `input_file`, `item_reference`, video inputs, and `input_image.file_id`
* provider-hosted tools; function and custom tools remain supported
* NanoGPT-only features such as Advisor, memory, scraping, retention overrides, provider or BYOK controls, caching controls, and billing overrides

Remote HTTP(S) image URLs and image data URLs are accepted. Structured output through the Responses `text.format` field is supported.

## Billing

Batch jobs use NanoGPT account balance and do not use subscription included tokens. At creation, NanoGPT checks the balance against a conservative maximum-liability estimate based on the input and output caps. Completed usage is charged once after the batch reaches a terminal state. If there is no billable usage, no usage charge is created.

Supported batch token usage is priced 50% below the equivalent synchronous request. Non-token charges, where supported, keep their normal rate. Use the live pricing page or pricing API as the source of truth.

## Common errors

* **Unsupported endpoint:** use `/v1/chat/completions` or `/v1/responses`, consistently across all rows.
* **Missing output cap:** add `max_tokens` or `max_completion_tokens` for Chat Completions, or `max_output_tokens >= 16` for Responses.
* **Mixed model:** every row must use the same model.
* **Streaming unsupported:** remove `stream: true`.
* **Unsupported Responses model:** choose a direct OpenAI model supported by the upstream Batch API.
* **Unsupported Responses field:** remove stateful features, provider-hosted tools, file references, or NanoGPT-only extensions.