Set up the local proxy
Use Node.js 22.23.4, 24.22.1 or 26.12.1, or a later supported release. Older runtimes are refused because of known security vulnerabilities. SetNANOGPT_API_KEY in your local environment, then start an explicitly selected release:
http://127.0.0.1:8787/v1. Keep your NanoGPT key in the proxy process; the calling application can use unused as its API key.
stream returns a single JSON response. Long reasoning requests may take more than five minutes; configure the calling application’s timeout accordingly.
Choose NanoGPT’s upstream host
The default upstream host ishttps://nano-gpt.com. Set --api-base to change the host contacted by the proxy:
/api or /v1. The proxy appends its own /api/v1/private/tinfoil/* paths. Your application’s base URL remains http://127.0.0.1:8787/v1. See API Hosts for host differences.
The proxy permits same-machine clients only and rejects browser origins by default. If a local browser application needs access, allow its exact origin:
Updates and model discovery
The proxy does not update itself or download new model mappings. Review releases before explicitly installing another version. The npm release includes a shrinkwrap that pins its runtime dependencies for registry installs; see the package README for persistent installation and lockfile guidance. The model list is bundled with the installed release. At startup, the proxy checks the hosted catalog and warns about additions or removals. Retired models are disabled. If the catalog cannot be fetched, the proxy warns and uses its bundled list; requests still require NanoGPT preflight and verified attestation.Supported models
Release 0.2.31 bundles these model IDs. Use the local/v1/models endpoint to check availability for your installed release.
Keep the same private model ID when enabling reasoning. Kimi K3.1 Pro and GLM 5.3 always generate reasoning; GLM 5.3 accepts
low, high or max effort. DeepSeek V4.1 Flash and Gemma 4 support optional thinking. reasoning: { "exclude": true } hides reasoning in the response without disabling its generation.
Model context windows include both input and output. The context_length and max_output_tokens values returned by the model list are not additive.
Attestation and privacy boundaries
The proxy verifies the router’s hardware evidence, signed code release and measurements against its verifier policy before establishing encrypted transport. Failed or incomplete verification prevents inference. After five idle minutes by default, the next request creates a fresh verified transport; verification also has a freshness window. NanoGPT still sees account identity, selected model and TEE target metadata, timing, request and response sizes, status and usage metadata. Private Mode protects supported prompt and completion bodies from NanoGPT; it does not provide account anonymity. Your calling application and local proxy handle plaintext, so their logs, plugins, tools and device security remain part of your privacy boundary. Attestation covers Tinfoil’s router. It does not independently prove which model answered, usage correctness or prompt non-retention. Usage metadata remains trusted rather than end-to-end signed. See Private Mode verification details for the verification contract. The NanoGPT web app can produce a receipt that you can verify locally:Supported operations and limits
- Chat Completions: text generation, streaming and supported model features through the local
/v1/chat/completionsendpoint. The Responses API and Batch API are unavailable in Private Mode. - Vision: supported models accept inline image bytes inside the encrypted request. A remote image URL is not equivalent to sending an encrypted local image; check the model’s capabilities and the package guide.
- PDF text extraction: send exactly one PDF, up to 3 MiB, to the local
/v1/convert/fileendpoint with anX-NanoGPT-Private-Modelheader. This encrypted conversion returns Markdown and is billed separately from model tokens. A model’spdf_upload: falsedescribes native PDF prompts, not this conversion step. - Web search: unavailable. Search flags and native web-search tool declarations are rejected before inference. If your application runs external tools itself, those calls have their own privacy boundaries.
- Billing: a funded account and API-key spend-limit headroom are required. Streaming may temporarily reserve enough balance for up to 32,768 output tokens even when the encrypted request asks for fewer. Unused reserve is refunded after verified usage metadata is available; this is not the final charge.
- Duration: hosted generation has a 29-minute soft deadline within a 30-minute route limit. If a stream ends without a finish reason and the final
[DONE]marker, the proxy emitsprivate_stream_incomplete; preceding text may be partial. - Local prompt cache: the proxy’s cache-isolation key is held in memory. Restarting the proxy starts a new cache namespace and can increase input-token cost and time to first token.
Tor and the onion API
Private Mode is not currently available through NanoGPT’s onion gateway. The gateway does not forward the/api/v1/private/tinfoil/* routes and returns 404 onion_route_not_available. Setting --api-base to the onion address does not enable those routes or configure Tor transport.
Version 0.2.31 adds optional SOCKS5 transport with remote DNS. With your own Tor client already running, you can reach the regular HTTPS API through Tor:
verify. It covers the proxy process, not npm package downloads or your application’s own network requests. SOCKS URLs must use socks5h:// and cannot include credentials.
This does not enable NanoGPT’s onion gateway. When an HTTP .onion API base is configured, the SDK still requires HTTPS attestation bundles; the proxy obtains those from https://nano-gpt.com through SOCKS. Other verification evidence can also use HTTPS hosts through Tor exits. This is not an onion-only network path. Account identity remains visible to NanoGPT, and TLS certificate, hostname and attestation verification stay enabled. Check this page before attempting onion access; ordinary onion API support does not imply Private Mode support.