Model switching

Beta. Updated 2026-10-29.

One model is loaded at a time. A request for a model that is not loaded starts a model switch.

Example

curl -s https://citadel.albastov.com/v1/models -H "Authorization: Bearer $CITADEL_API_KEY" |
  python3 -c 'import json, sys; [print(m["id"], m["state"]) for m in json.load(sys.stdin)["data"]]'

Output

qwen3.8-27b loaded
qwen3.8-27b-fast stopped
qwen3.8-27b-uncensored stopped
qwen3.8-flash-next-abliterated stopped
deepseek-v4.1-flash stopped
unsloth/Qwen3.8-27B-GGUF/Qwen3.8-27B-UD-Q4_K_XL.gguf stopped

Switch

Requests on the loaded model finish, up to 120 s.
The loaded model stops.
The requested model starts. state is starting until the model answers.

Concurrent requests for the same model wait for one switch. Switches to different models run in sequence. A switch changes the loaded model for all callers.

Cold starts

Model	`cold_start_s`	Measured
`qwen3.8-27b`	150	226–285 s
`qwen3.8-27b-fast`	120	not measured
`qwen3.8-27b-uncensored`	180	not measured
`qwen3.8-flash-next-abliterated`	240	not measured
`deepseek-v4.1-flash`	300	not measured
llama.cpp model	30	not measured

Measured 2026-10-29. qwen3.8-27b took 656 s on the first start after an engine update. cold_start_s is the estimate in GET /v1/models. A switch from qwen3.8-flash-next-abliterated or deepseek-v4.1-flash adds time. A loaded model unloads after 30 minutes without requests.

Requests during a load

Request	Result
Streamed, no `X-LLM-Wait`	Waits up to 15 min; `: loading` comments from 10 s
Not streamed, no `X-LLM-Wait`	Retried by Citadel for 80 s, then `503 model_loading`
`X-LLM-Wait: N`	Waits up to N s (maximum 900), then `503 model_loading`
`X-LLM-Wait: 0`	`503 model_loading` at once; the switch continues

Not streamed with X-LLM-Wait above 90: Cloudflare ends the request at about 100 s with 524.

503 model_loading, Retry-After: 150

{
  "error": {
    "message": "Loading qwen3.8-27b; retry in 150 s",
    "type": "model_loading",
    "code": "model_loading",
    "model": "qwen3.8-27b",
    "state": "starting",
    "retry_after_s": 150
  }
}

Retry-After and retry_after_s equal cold_start_s minus the elapsed load time, minimum 5 s. After 15 minutes of loading, both are 30.

Waiting

Interactive requests: stream without X-LLM-Wait.
Batch jobs: send X-LLM-Wait: 0. Poll GET /v1/models every 10 to 30 s until state is loaded. Then send the batch.
SDK retries: the OpenAI Python SDK (3.x) retries a 503 twice and follows Retry-After up to 120 s. Set max_retries=0 and use a retry loop: 503 retries.