An experimental sycophancy-ablated version of Mistral AI's Mistral-Medium-3.5-128B, produced with heretic + ARA-LoRA.

This is a text-only model. The vision tower has been stripped to save VRAM.

The entire abliteration pipeline was run in under 48GB VRAM on 2x3090s. To start, the vision tower was stripped from Mistral-Medium-3.5-128B. It was then quantized to 2.50bpw with exllamav3, then ARA-LoRA abliteration was performed on this quantized model, using my fork of heretic with exl3 support and the ARA-LoRA PR. The resulting LoRA was then merged back to the original fp8 weights, producing the model in this repo.

An interesting result is how cheap the edit is relative to the compression. The abliteration perturbs the model by KL 0.0994 against the fp8 base, well below the quantization costs for this model, which are abnormally high to begin with. The 2.5bpw has a KL div of 0.4274, and a 2.25bpw quant lands at 0.5566. The abliteration costs roughly a quarter of what the compression does, and perplexity barely moves (4.94 → 5.20 on the merged weights, well below where the quants sit).

Mistral Medium is considerably more sycophantic than gemma to start with: 33/88 markers under the sycophancy-induction prompt, versus gemma's 10/88. That gives the ablation a lot more to work with, and the drop to 5/88 is the more dramatic for it. The effect appears bi-directional; inverting the LoRA amplifies sycophancy rather than suppressing it (see the demo below).

Another thing worth noting, the instruction-following ability of this model can still overpower the abliteration. So with the sycophancy-induction system prompt, the model still behaves somewhat sycophantically. But with a normal system prompt, the sycophancy is greatly reduced.

This is part of an ongoing effort to find behaviors other than refusal for heretic to modify, and to extend heretic support to exl3-quantized models.

An exl3 quantization is here.
The fp8 weights are here.