Chat completions

A chat completion is a model's reply to a list of messages.

Send messages

messages is the conversation so far, oldest first. Each message has a role and text content.

Role	Content
`system`	Instructions for the whole conversation; first in the list
`user`	The caller's turn
`assistant`	An earlier reply from the model
`tool`	A tool result, with the `tool_call_id` of the call (Tools and structured output)

curl

curl https://api.albastov.com/v1/chat/completions \
  -H "Authorization: Bearer $ALBASTOV_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3.8-27b",
    "max_tokens": 100,
    "messages": [
      {"role": "system", "content": "Answer in one line."},
      {"role": "user", "content": "Give me three names for a grey cat."}
    ],
    "chat_template_kwargs": {"enable_thinking": false}
  }'

Python

import os
from openai import OpenAI

client = OpenAI(base_url="https://api.albastov.com/v1", api_key=os.environ["ALBASTOV_API_KEY"])
reply = client.chat.completions.create(
    model="qwen3.8-27b",
    max_tokens=100,
    messages=[
        {"role": "system", "content": "Answer in one line."},
        {"role": "user", "content": "Give me three names for a grey cat."},
    ],
    extra_body={"chat_template_kwargs": {"enable_thinking": False}},
)
print(reply.choices[0].message.content)

JavaScript

const res = await fetch("https://api.albastov.com/v1/chat/completions", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.ALBASTOV_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "qwen3.8-27b",
    max_tokens: 100,
    messages: [
      { role: "system", content: "Answer in one line." },
      { role: "user", content: "Give me three names for a grey cat." },
    ],
    chat_template_kwargs: { enable_thinking: false },
  }),
});
if (!res.ok) throw new Error(`${res.status} ${await res.text()}`);
const reply = await res.json();
console.log(reply.choices[0].message.content);

The answer is choices[0].message.content. finish_reason is stop for a complete answer, length when max_tokens cut the answer, and tool_calls for a tool call. usage counts prompt and output tokens.

Conversation history

The API keeps no conversation state. Send the whole conversation on every request: append the model's reply as an assistant message, then the next user message.

Python

import os
from openai import OpenAI

client = OpenAI(base_url="https://api.albastov.com/v1", api_key=os.environ["ALBASTOV_API_KEY"])
no_thinking = {"chat_template_kwargs": {"enable_thinking": False}}

history = [{"role": "user", "content": "Explain prefix caching in two sentences."}]
reply = client.chat.completions.create(model="qwen3.8-27b", messages=history, max_tokens=120, extra_body=no_thinking)
print(reply.choices[0].message.content)

history += [{"role": "assistant", "content": reply.choices[0].message.content},
            {"role": "user", "content": "Now in one."}]
reply = client.chat.completions.create(model="qwen3.8-27b", messages=history, max_tokens=60, extra_body=no_thinking)
print(reply.choices[0].message.content)

A conversation longer than the model's context length fails. Context lengths: Chat models.

OpenAI SDKs

Set base_url to https://api.albastov.com/v1 and api_key to an Albastov API key with the scope llm. chat_template_kwargs is not an OpenAI parameter: the Python SDK sends the field through extra_body; curl and fetch send the field at the top level of the body.

Python

import os
from openai import OpenAI

client = OpenAI(base_url="https://api.albastov.com/v1", api_key=os.environ["ALBASTOV_API_KEY"])
reply = client.chat.completions.create(
    model="qwen3.8-27b",
    messages=[{"role": "user", "content": "Name the largest moon of Saturn."}],
    max_tokens=50,
    extra_body={"chat_template_kwargs": {"enable_thinking": False}},
)
print(reply.choices[0].message.content)

JavaScript

import OpenAI from "openai";

const client = new OpenAI({ baseURL: "https://api.albastov.com/v1", apiKey: process.env.ALBASTOV_API_KEY });
const reply = await client.chat.completions.create({
  model: "qwen3.8-27b",
  messages: [{ role: "user", content: "Name the largest moon of Saturn." }],
  max_tokens: 50,
  chat_template_kwargs: { enable_thinking: false },
});
console.log(reply.choices[0].message.content);

Prompt caching

A request whose messages start with the same text as an earlier request reuses the processed prefix, and the first token arrives sooner. usage.prompt_tokens_details.cached_tokens counts the reused prompt tokens.

Resend earlier messages unchanged: same order, same text. Append new turns at the end.
Put the fixed system prompt first, and per-request content (dates, retrieved documents) after the fixed part.
Send earlier assistant turns with content only, without reasoning.
Loading another model clears the cache. The cache is shared by all callers; the oldest entries leave first.

Parallel requests

An account runs up to capabilities.max_parallel requests at once on a model (64 on qwen3.8-27b). A request above the limit fails with 400 too_many_parallel, or 400 parallel_unsupported on a model that runs one request at a time. The example sends eight conversations at once.

Python

import asyncio, os
from openai import AsyncOpenAI

client = AsyncOpenAI(base_url="https://api.albastov.com/v1", api_key=os.environ["ALBASTOV_API_KEY"])
topics = ["rivers", "volcanoes", "glaciers", "deserts", "tides", "comets", "auroras", "monsoons"]

async def ask(topic):
    reply = await client.chat.completions.create(
        model="qwen3.8-27b",
        messages=[{"role": "user", "content": f"In two sentences, what are {topic}?"}],
        max_tokens=120,
        extra_body={"chat_template_kwargs": {"enable_thinking": False}},
    )
    return topic, reply.choices[0].message.content

async def main():
    for topic, answer in await asyncio.gather(*(ask(t) for t in topics)):
        print(topic, "-", answer)

asyncio.run(main())

Retries

503 with model_loading, queued, gpu-busy or device_busy carries Retry-After in seconds. Wait Retry-After seconds and send the same request again. Branch on error.code. A 429 daily limit resets at 00:00 UTC; retrying earlier fails again. The OpenAI Python SDK retries a 503 twice by default: set max_retries=0 and retry in a loop.

Python

import os, time
from openai import OpenAI, APIStatusError

client = OpenAI(base_url="https://api.albastov.com/v1", api_key=os.environ["ALBASTOV_API_KEY"], max_retries=0)
RETRY = {"model_loading", "queued", "gpu-busy", "device_busy"}

def create(deadline_s=900, **kwargs):
    start = time.monotonic()
    while True:
        try:
            return client.chat.completions.create(**kwargs)
        except APIStatusError as e:
            code = e.body.get("code") if isinstance(e.body, dict) else None
            wait = int(e.response.headers.get("Retry-After", "10"))
            if e.status_code != 503 or code not in RETRY or time.monotonic() - start + wait > deadline_s:
                raise
            print(f"503 {code}, retrying in {wait} s")
            time.sleep(wait)

reply = create(model="qwen3.8-27b", max_tokens=60,
               messages=[{"role": "user", "content": "One sentence on patience."}],
               extra_body={"chat_template_kwargs": {"enable_thinking": False}})
print(reply.choices[0].message.content)

JavaScript

const RETRY = new Set(["model_loading", "queued", "gpu-busy", "device_busy"]);

async function create(body, deadlineMs = 900_000) {
  const start = Date.now();
  for (;;) {
    const res = await fetch("https://api.albastov.com/v1/chat/completions", {
      method: "POST",
      headers: {
        Authorization: `Bearer ${process.env.ALBASTOV_API_KEY}`,
        "Content-Type": "application/json",
        "X-LLM-Wait": "0", // 503 at once; the loop waits
      },
      body: JSON.stringify(body),
    });
    if (res.ok) return res.json();
    const err = (await res.json().catch(() => ({}))).error ?? {};
    const waitS = Number(res.headers.get("Retry-After") ?? 10);
    if (res.status !== 503 || !RETRY.has(err.code) || Date.now() - start + waitS * 1000 > deadlineMs) {
      throw new Error(`${res.status} ${err.code ?? ""} ${err.message ?? ""}`);
    }
    console.log(`503 ${err.code}, retrying in ${waitS} s`);
    await new Promise((r) => setTimeout(r, waitS * 1000));
  }
}

const reply = await create({
  model: "qwen3.8-27b",
  max_tokens: 60,
  messages: [{ role: "user", content: "One sentence on patience." }],
  chat_template_kwargs: { enable_thinking: false },
});
console.log(reply.choices[0].message.content);

Model loading and the queue: Language models.

Reference

Chat completions reference: every parameter, header, response field and error.