Quantization and abliteration
Beta. Updated 2026-10-29.
Quantization
Quantization stores model weights in fewer bits than the 16-bit original. A quantized model uses less memory and generates faster, at lower accuracy.
| Model | Format | Bits | Size |
|---|---|---|---|
qwen3.8-27b | W4A16 AutoRound | 4 | about 14 GB |
qwen3.8-27b-fast | W4A16 AutoRound | 4 | about 14 GB |
qwen3.8-27b-uncensored | W4A16 AutoRound | 4 | about 14 GB |
qwen3.8-flash-next-abliterated | IQ2_XS | about 2 | not measured |
deepseek-v4.1-flash | 2.00 bpw | 2 | about 138 GB |
| llama.cpp model | Q4_K_XL GGUF | 4 | not measured |
16-bit sizes: Qwen3.8 27B about 54 GB, DeepSeek V4.1 Flash about 1.1 TB.
| Precision | Accuracy cost |
|---|---|
| 4-bit | Small; first visible in exact recall, long arithmetic, long code |
| 2-bit | DeepSeek V4.1 Flash: perplexity +15.5 % (quantizer figure) |
Accuracy costs are not measured on these deployments. Check exact output (numbers, code, citations) from the 2-bit models.
qwen3.8-flash-next-abliterated and deepseek-v4.1-flash are mixture-of-experts models. deepseek-v4.1-flash
weights exceed RAM and VRAM; experts load from disk during generation, which limits speed.
Abliteration
Abliteration removes the refusal direction from a model's activations. An abliterated model does not refuse requests and does not add safety warnings.
| Model | Method |
|---|---|
qwen3.8-27b-uncensored | Abliterated community checkpoint |
qwen3.8-flash-next-abliterated | Run-time projection, layers 4 to 44 |
qwen3.8-27b-uncensored checkpoint: Swift-Qwen3.8-27B-Uncensored-W4A16-AutoRound.
| Property | Value |
|---|---|
| Accuracy cost | Slightly lower instruction following and accuracy; not measured |
| Knowledge | None added |
| Access | Any LLM key; any signed-in member session |
Abliterated output has no safety filtering. Moderate the output before showing it to other people.