Chat completions

Generate a model response for a list of messages.

Preview: every request currently returns the fixed reply "Hello world!". Model generation is not connected yet, and billing and usage are already real.

Endpoint

endpoint
POST https://api.infere.dev/v1/chat/completions

Content type application/json. Authenticate with a bearer key.

Request body

FieldTypeRequiredDescription
modelstringyesA model id from the models page.
messagesarrayyesObjects with role (system, user, assistant) and content. The total must fit the context length of the model.
streambooleannoDefault false. When true, the response is sent as server-sent events.
max_tokensintegernoUpper bound on generated tokens. Billing counts only tokens actually generated.
temperaturenumbernoBetween 0 and 2. Default is the model default.
top_pnumbernoNucleus sampling, between 0 and 1.

Response

Without streaming, the response is one JSON object:

response.json
{  "id": "chatcmpl_8f2a91",  "model": "deepseek-v3",  "choices": [    {      "message": {        "role": "assistant",        "content": "KV caching stores attention keys and values so each new token reuses past work."      },      "finish_reason": "stop"    }  ],  "usage": { "prompt_tokens": 14, "completion_tokens": 19 }}

usage reports the tokens billed for the request. finish_reason is stop when the model finished, or length when max_tokens was reached.

Streaming

With stream: true the response is text/event-stream. Each event carries a delta. The last chunk before the end carries usage, and the stream ends with data: [DONE].

stream
data: {"id":"chatcmpl_8f2a91","object":"chat.completion.chunk","model":"deepseek-v3","choices":[{"index":0,"delta":{"role":"assistant","content":"KV caching"},"finish_reason":null}]}
data: {"id":"chatcmpl_8f2a91","object":"chat.completion.chunk","model":"deepseek-v3","choices":[{"index":0,"delta":{"content":" stores attention"},"finish_reason":null}]}
data: {"id":"chatcmpl_8f2a91","object":"chat.completion.chunk","model":"deepseek-v3","choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":14,"completion_tokens":19}}
data: [DONE]

Responses are not stored on our side; keep your own history if you need it. A request that fails before generation starts is not billed.