Training prompts: mlabonne/harmful_behaviors train[:400] / mlabonne/harmless_alpaca train[:400]
Evaluation harmful: mlabonne/harmful_behaviors test[:100]
Evaluation harmless: 100 custom German prompts (literary, factual, analytical, conversational)

Quantisations

All quants are split into ~32 GB shards using llama.cpp's native gguf-split format. llama.cpp loads them transparently — point at the first shard (-00001-of-NNNNN.gguf) and the rest is picked up automatically.

Type	Filename	Size
f16	Part 1 Part 2 Part 3 Part 4 Part 5 Part 6 Part 7 Part 8	245 GB
Q8_0	Part 1 Part 2 Part 3 Part 4 Part 5	130 GB
Q6_K	Part 1 Part 2 Part 3 Part 4	100 GB
Q5_K_M	Part 1 Part 2 Part 3	86 GB
Q4_K_M	Part 1 Part 2 Part 3	73 GB

Quality recommendation: Q6_K offers ~99% of f16 quality at 41% the size. Q4_K_M is the smallest with negligible measurable degradation on German output, runs comfortably on a single 96 GB card.

Inference

The model uses Mistral's standard [INST]...[/INST] prompt format. The GGUF tokenizer correctly preserves [INST], [/INST], [TOOL_CALLS] and the related Mistral special-token IDs (3-15) as single-token IDs — which the upstream tokenizer_config.json ships with special: false, breaking GGUF tokenization in earlier quants. Fixed via tokenizer-config patch before convert.

Notes

KL divergence is computed on a custom German prompt set, which means the optimizer specifically protects German output quality during ablation. The model retains its base capabilities for narrative writing, factual queries, and conversational German.

License

Inherits the Mistral Research License from the base model.

Frisch12/magnum-v4-123b-heretic-DE-GGUF

magnum-v4-123b-heretic-DE-GGUF

Heretic results

Configuration

Quantisations

Inference

Notes

License

Frisch12/
magnum-v4-123b-heretic-DE-GGUF