A legal-domain fine-tune of
huihui-ai/Huihui-Qwen3.5-9B-abliterated
(an abliterated Qwen3.5-9B vision-language model), trained on public-domain United States court
opinions from
CourtListener. Smaller, faster sibling of
Legal-Qwen3.6-27B-Abliterated.
All base capabilities are preserved and were verified after merging: vision, tool/function
calling, and thinking mode.
Fine-tuning teaches doctrine and style, not verbatim recall. The model can hallucinate reporter
citations, dates, and quotes. For real legal-research use, run it with RAG over the
CourtListener bulk data (or the linked
training corpus) and
verify every citation at the source.
Speed notes from testing on an RTX 5090 (single-stream): ~47 tok/s bf16 in vLLM with CUDA graphs
off, 87 tok/s with graphs on, 146 tok/s with an FP8 W8A8 quant (llm-compressor FP8_DYNAMIC,
recipe in this card's discussion) plus graphs; ~104 tok/s as Q8_0 in Ollama. Installing the
Triton-based flash-linear-attention package speeds up the hybrid attention layers in vLLM.
