dful.
Dful Dark
Theme
Knowledge tree
On this page

Model APIs

The basic shape of calling a model from code and handling output, usage, streaming, tools, and state.
Updated 28 Aug 2026

An application normally uses a model in this sequence:

  1. Call an endpoint.
  2. Specify the model, input, and a few parameters.
  3. Receive output.
  4. Read metadata such as usage.
  5. Optionally handle streaming or tool calls.

The API defines how those pieces are represented. The model performs the inference.

LM Studio example

LM Studio can expose a local model through OpenAI-compatible endpoints. An OpenAI SDK client can use it by changing base_url:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:1234/v1",
    api_key="lm-studio",
)

response = client.chat.completions.create(
    model="model-identifier",
    messages=[
        {"role": "user", "content": "Summarize the changes in this diff."}
    ],
    temperature=0.2,
)

print(response.choices[0].message.content)
print(response.usage)

model-identifier must match the loaded model. The client sends the request to LM Studio, LM Studio runs inference, and the response uses the shape expected by the SDK.

This example uses Chat Completions because its messages structure is common in local clients. LM Studio also documents /v1/responses and its own REST API.

Request and response

The request needs a model identifier and input. It may also set output limits, sampling parameters, or tool definitions.

A response is not always just text. It can include a stop reason, usage, or a tool call. Client code should read the documented response structure instead of assuming that one visible text field is the complete result.

Usage normally includes input and output token counts. It is the best starting point for understanding what the call processed and calculating cost. Additional categories depend on the provider.

Streaming

With streaming, output arrives as events or fragments. The client can display text while it is generated instead of waiting for completion.

It may also need to assemble tool-call arguments delivered across several events. Final usage is commonly available near the end of the stream, depending on the API.

Streaming changes delivery. The inference stages remain prefill and decode.

Tools

A request can include tool definitions. Instead of returning final text, the model may ask to call one function with particular arguments.

The application validates and executes the operation, then supplies the result so generation can continue. Tool Use covers that path. At API level, the client must preserve the IDs and objects that associate a call with its result.

Conversation state

Some APIs require relevant messages to be sent again on every request. Others let a request reference an earlier response or conversation.

Application state can still live elsewhere in either case: a database, repository, or agent harness. Provider-managed continuation does not make that conversation the only source of task state.

OpenAI-compatible

OpenAI-compatible normally means another server implements OpenAI-shaped endpoints or formats so existing clients can be reused.

It does not guarantee identical behavior. A server may accept messages and temperature while differing in available models, tools, reasoning controls, usage, or ignored parameters. Test the capabilities the integration needs, not only whether one simple request succeeds.

References