Skip to content

Tools and structured output

Beta. Updated 2026-10-29.

Function calling and JSON output in the OpenAI format.

Tool calls

  1. Send tools. The response has finish_reason: "tool_calls" and message.tool_calls.
  2. Run the function. Parse function.arguments (a JSON string) and validate the values.
  3. Send the assistant message and a tool message with the same tool_call_id. The response contains the answer.
tool_choiceResult
"auto"Model decides; default with tools
"none"No tool calls
"required"At least one tool call
{"type": "function", "function": {"name": "..."}}A call to the named function

Set "chat_template_kwargs": {"enable_thinking": false} for faster tool calls.

Python

import json, os
from openai import OpenAI

client = OpenAI(base_url="https://citadel.albastov.com/v1", api_key=os.environ["CITADEL_API_KEY"])
tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Current weather for a city",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}]
messages = [{"role": "user", "content": "What's the weather in Oslo?"}]
no_thinking = {"chat_template_kwargs": {"enable_thinking": False}}

first = client.chat.completions.create(model="qwen3.8-27b", messages=messages, tools=tools, extra_body=no_thinking)
call = first.choices[0].message.tool_calls[0]
print(first.choices[0].finish_reason, call.function.name, call.function.arguments)

args = json.loads(call.function.arguments)
result = {"city": args["city"], "temp_c": 7, "sky": "overcast"}  # your function runs here

messages.append(first.choices[0].message.model_dump(exclude_none=True))
messages.append({"role": "tool", "tool_call_id": call.id, "content": json.dumps(result)})
final = client.chat.completions.create(model="qwen3.8-27b", messages=messages, tools=tools, extra_body=no_thinking)
print(final.choices[0].message.content)

curl

TOOLS='[{"type": "function", "function": {"name": "get_weather", "description": "Current weather for a city",
  "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}}}]'

# 1. Request with tools: the response contains a tool call.
curl -s https://citadel.albastov.com/v1/chat/completions \
  -H "Authorization: Bearer $CITADEL_API_KEY" -H "Content-Type: application/json" \
  -d '{"model": "qwen3.8-27b", "tools": '"$TOOLS"',
       "messages": [{"role": "user", "content": "What is the weather in Oslo?"}],
       "chat_template_kwargs": {"enable_thinking": false}}' > first.json
python3 -c 'import json; m = json.load(open("first.json"))["choices"][0]["message"]; print(m["tool_calls"][0]["function"])'

# 2. Send the tool result with the tool call id: the response contains the answer.
CALL_ID=$(python3 -c 'import json; print(json.load(open("first.json"))["choices"][0]["message"]["tool_calls"][0]["id"])')
curl -s https://citadel.albastov.com/v1/chat/completions \
  -H "Authorization: Bearer $CITADEL_API_KEY" -H "Content-Type: application/json" \
  -d '{"model": "qwen3.8-27b", "tools": '"$TOOLS"',
       "messages": [
         {"role": "user", "content": "What is the weather in Oslo?"},
         {"role": "assistant", "content": null, "tool_calls": [{"id": "'"$CALL_ID"'", "type": "function",
           "function": {"name": "get_weather", "arguments": "{\"city\": \"Oslo\"}"}}]},
         {"role": "tool", "tool_call_id": "'"$CALL_ID"'", "content": "{\"temp_c\": 7, \"sky\": \"overcast\"}"}],
       "chat_template_kwargs": {"enable_thinking": false}}'
rm first.json

JSON output

response_format constrains generation (vLLM guided decoding).

response_format.typeOutput
json_objectA JSON object
json_schemaJSON that matches json_schema.schema
response_format, json_schema
{"type": "json_schema", "json_schema": {"name": "language", "schema": {"type": "object"}, "strict": true}}

With json_object, list the required keys in the prompt.

curl

curl https://citadel.albastov.com/v1/chat/completions \
  -H "Authorization: Bearer $CITADEL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "qwen3.8-27b",
       "messages": [{"role": "user", "content": "Name one programming language and the year it appeared."}],
       "response_format": {"type": "json_schema", "json_schema": {"name": "language", "strict": true,
         "schema": {"type": "object", "additionalProperties": false, "required": ["name", "year"],
           "properties": {"name": {"type": "string"}, "year": {"type": "integer"}}}}},
       "chat_template_kwargs": {"enable_thinking": false}}'

Python

import json, os
from openai import OpenAI

client = OpenAI(base_url="https://citadel.albastov.com/v1", api_key=os.environ["CITADEL_API_KEY"])
schema = {
    "type": "object",
    "additionalProperties": False,
    "required": ["name", "year"],
    "properties": {"name": {"type": "string"}, "year": {"type": "integer"}},
}
reply = client.chat.completions.create(
    model="qwen3.8-27b",
    messages=[{"role": "user", "content": "Name one programming language and the year it appeared."}],
    response_format={"type": "json_schema", "json_schema": {"name": "language", "schema": schema, "strict": True}},
    extra_body={"chat_template_kwargs": {"enable_thinking": False}},
)
language = json.loads(reply.choices[0].message.content)
print(language["name"], language["year"])

Limits

  • Supported on qwen3.8-27b, qwen3.8-27b-fast and qwen3.8-27b-uncensored. Not verified on the other models.
  • Output cut by max_tokens (finish_reason: "length") is invalid JSON. Set max_tokens above the expected output length.
  • The schema applies to content, not to reasoning.
  • Streamed tool calls arrive as delta.tool_calls fragments. Concatenate function.arguments per index until finish_reason. The OpenAI SDKs concatenate the fragments.