Model APIs

The request and response behind a model integration, with practical handling of usage, streaming, tool calls, and OpenAI-compatible endpoints.

Updated 6 Oct 2026

From code, calling a model usually means building a request with the model, input, and required parameters, sending it to an endpoint, and processing the response. Usage, streaming, and tool calls build on that basic exchange.

LM Studio example

LM Studio can expose a local model through OpenAI-compatible endpoints. This lets an OpenAI SDK client be reused by changing base_url:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:1234/v1",
    api_key="lm-studio",
)

response = client.chat.completions.create(
    model="model-identifier",
    messages=[
        {"role": "user", "content": "Summarize the changes in this diff."}
    ],
    temperature=0.2,
)

print(response.choices[0].message.content)
print(response.usage)

model-identifier must match the loaded model. The client sends the request to LM Studio, the runtime performs inference, and the server returns a structure compatible with the SDK.

This example uses Chat Completions because its messages shape is still common in local clients. LM Studio also documents /v1/responses and its own REST API.

Request and response

A request identifies the model and provides input. Depending on the API, it can also include output limits, sampling parameters, instructions, or tool definitions.

A response may contain text, a tool call, a stop reason, and metadata such as usage. Client code should read the documented endpoint structure rather than assume one visible text field represents the complete result.

Usage commonly separates input and output tokens and is the starting point for understanding how much the call processed and what it cost. Additional categories such as cached input or reasoning depend on the provider.

Streaming

With streaming, output arrives as events or fragments while generation is in progress. The client can display text incrementally and, where supported, assemble a tool call whose arguments arrive across several events.

Final usage commonly appears at the end of the stream or in a dedicated event. The exact shape depends on the provider protocol.

Tool calls

A request can include tool definitions. Instead of producing a final text answer, the model may return an operation name and structured arguments.

The application validates and executes that operation, then supplies the result so the model can continue. Tool Use covers that path; at API level, the client must preserve the IDs or objects that associate each call with its result.

State

Some APIs expect the application to resend relevant history on each request. Others let a request refer to an earlier response or conversation.

That continuation mechanism does not replace application state. A repository, database, or agent harness can still be the source of truth for completed work, artifacts, and decisions that need to persist.

OpenAI-compatible

OpenAI-compatible usually means another server implements endpoints or formats close enough to OpenAI’s shape for existing clients to be reused.

Compatibility should be tested against the features the integration actually needs. A server may accept messages and temperature while differing in tools, reasoning controls, usage, streaming, or ignored parameters.