LLM API

Beta. Updated 2026-10-30.

OpenAI-compatible chat completions on open-weight models.

Quick start

export CITADEL_API_KEY=hlk_...

curl

curl https://citadel.albastov.com/v1/chat/completions \
  -H "Authorization: Bearer $CITADEL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3.8-27b",
    "messages": [{"role": "user", "content": "Say hello in five words."}],
    "chat_template_kwargs": {"enable_thinking": false},
    "max_tokens": 50
  }'

Python

import os
from openai import OpenAI

client = OpenAI(base_url="https://citadel.albastov.com/v1", api_key=os.environ["CITADEL_API_KEY"])
reply = client.chat.completions.create(
    model="qwen3.8-27b",
    messages=[{"role": "user", "content": "Say hello in five words."}],
    extra_body={"chat_template_kwargs": {"enable_thinking": False}},
    max_tokens=50,
)
print(reply.choices[0].message.content)

JavaScript

const res = await fetch("https://citadel.albastov.com/v1/chat/completions", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.CITADEL_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "qwen3.8-27b",
    messages: [{ role: "user", content: "Say hello in five words." }],
    chat_template_kwargs: { enable_thinking: false },
    max_tokens: 50,
  }),
});
if (!res.ok) throw new Error(`${res.status} ${await res.text()}`);
const reply = await res.json();
console.log(reply.choices[0].message.content);

Response

{
  "id": "chatcmpl-b50af0eb9c5d02d9",
  "object": "chat.completion",
  "created": 1793305215,
  "model": "qwen3.8-27b",
  "choices": [
    {
      "index": 0,
      "message": {"role": "assistant", "content": "Hello there, my friend.", "reasoning": null},
      "finish_reason": "stop"
    }
  ],
  "system_fingerprint": "vllm-0.31.0-da5a7807",
  "usage": {
    "prompt_tokens": 18,
    "completion_tokens": 7,
    "total_tokens": 25,
    "prompt_tokens_details": {"cached_tokens": 0},
    "completion_tokens_details": {"reasoning_tokens": 0}
  }
}

A request for a model that is not loaded returns 503 model_loading with Retry-After. See Model switching.

Reference

Endpoint	Returns
`GET /v1/models`	Installed models, with `state` and `context_length`
`POST /v1/chat/completions`	Chat completion, streamed or not
`GET /v1/llm/docs`	LLM API page as markdown; no key

Item	Value
Base URL	`https://citadel.albastov.com/v1`
API	OpenAI chat completions; OpenAI SDKs with `base_url`
Default model	`qwen3.8-27b`
Models loaded at once	1
Hardware	RTX 3090 (24 GB VRAM), 64 GB RAM
Prompt and output storage	none

Parameters, response fields and errors: Chat completions.

Models

Model	Context	Speed	Concurrency
`qwen3.8-27b`	150,000	41 tok/s	64
`qwen3.8-27b-fast`	65,536	100–157 tok/s (engine figure)	8
`qwen3.8-27b-uncensored`	150,000	not measured	64
`qwen3.8-flash-next-abliterated`	131,072	87 tok/s (vendor figure)	1
`deepseek-v4.1-flash`	8,192	not measured	1

Speed: output tokens per second for one request, measured 2026-10-29 (streamed, thinking off, 200 output tokens). Concurrency: requests one model runs at once.

Model	Base model	Quantization	Abliterated
`qwen3.8-27b`	Qwen3.8 27B	4-bit	no
`qwen3.8-27b-fast`	Qwen3.8 27B	4-bit	no
`qwen3.8-27b-uncensored`	Qwen3.8 27B Swift	4-bit	yes
`qwen3.8-flash-next-abliterated`	Qwen3.8 Flash-Next	2-bit	yes
`deepseek-v4.1-flash`	DeepSeek V4.1 Flash, 552B	2-bit	no

GET /v1/models also lists unsloth/Qwen3.8-27B-GGUF/Qwen3.8-27B-UD-Q4_K_XL.gguf: Qwen3.8 27B on llama.cpp, 4-bit, not measured. Bits and abliteration: Quantization and abliteration.

Limits

One model is loaded at a time. A request for another model starts a 2 to 5 minute model switch (Model switching).
A model unloads after 30 minutes without requests.
A request for a model that is not loaded can wait up to 900 s in the GPU queue (Queueing).
Non-streamed requests end after about 100 s with 524. Stream long requests.
The API is stateless. Send the full conversation on every request.
An LLM key is limited to 200 calls per UTC day by default (Rate limits).
Text only. No embeddings, image input, audio or legacy completions.

Known issues

2026-10-29: With 30 conversations of about 2,400 tokens, the second turn gets no cached tokens.
2026-10-29: The first start after an engine update takes about 11 minutes.
2026-10-29: Cloudflare rejects the Python urllib default User-Agent with 403.

Workarounds: Known issues.

Examples

Changelog

2026-10-30: Changed: an image or music job that has waited 300 s behind a steady stream of requests goes next. New requests for the loaded model wait behind it (503 queued with X-LLM-Wait: 0), then the model loads again. See Queueing.
2026-10-29: Fixed: qwen3.8-27b stall under the first heavy load after a start. Cause: GPU memory overcommit. The engine starts with 0.90 of GPU memory (was 0.91); prefix cache 6.35 GiB (was 6.6 GiB). The stall watchdog (502 engine_restarted) remains.
2026-10-29: A request waiting for the GPU returns 503 queued with position (with X-LLM-Wait: 0). 503 gpu-busy only after 900 s in the queue.
2026-10-29: Docs moved to albastov.com/docs. GET /v1/llm/docs returns the LLM API page. The reasoning field is reasoning (vLLM 0.31), not reasoning_content.
2026-10-29: qwen3.8-27b stall watchdog: 502 engine_restarted.
2026-10-29: API released: key and session routes, daily key limits, streamed usage, loading comments.