This is the
W4A16 GPTQ-quantized version of
tacodevs/Behemoth-T1-123B — a 123B Mistral Large roleplay model that
thinks like a literary author before it writes like a storyteller.
For the full pitch, training details, and the philosophy behind T1, see the
BF16 model card.
1import openai
2
3client = openai.OpenAI(base_url="http://localhost:8000/v1", api_key="-")
4
5PREFILLS = {
6 "analytical": "Ok i need to think about how to respond — what does the character feel right now, what from their experience is relevant, what do they value, and what are they trying to achieve, so",
7 "creative": "Ok i need to think as a creative writer — what twist would surprise here? Let me find an engaging new direction nobody saw coming, so",
8 "unhinged": "Ok i need to think as an unhinged author — raw, explicit, intense, fully in character with no holding back, so",
9}
10
11response = client.chat.completions.create(
12 model="tacodevs/Behemoth-T1-123B-GPTQ",
13 messages=[
14 {"role": "system", "content": CHARACTER_CARD},
15 *conversation_history,
16 {"role": "user", "content": user_message},
17 {"role": "assistant", "content": f"<think>\n{PREFILLS['creative']}\n"},
18 ],
19 extra_body={
20 "continue_final_message": True,
21 "add_generation_prompt": False,
22 },
23 temperature=0.6,
24 max_tokens=2048,
25 stop=["[INST]", "</s>"],
26)
1vllm serve tacodevs/Behemoth-T1-123B-GPTQ \
2 --tokenizer-mode auto \
3 --max-model-len 8192 \
4 --gpu-memory-utilization 0.9
Single 80 GB H100, single 80 GB A100, or single 96 GB H100 NVL all fit comfortably.
W4A16 quantization has measurable but small impact on the literary thinking pattern T1 was trained for:
For most users, GPTQ T1 is indistinguishable from BF16 T1 in normal use. Only A/B testing with the same seed would expose the differences.
If you want maximum quality and have the VRAM, use the
BF16 reference or
FP8 W8A8 (~99% of BF16, fits on 2×80 GB).
T1 is a LoRA distillation of Claude Opus 4.5 literary thinking onto
tacodevs/Behemoth-X-R1-123B
(itself an SCE merge of Behemoth-X creative writing + Behemoth-R1 reasoning).
1@misc{behemoth-t1-2026,
2 title = {Behemoth-T1-123B: Literary Thinking Distillation for RP},
3 author = {tacodevs},
4 year = {2026},
5 url = {https://huggingface.co/tacodevs/Behemoth-T1-123B},
6}
