LFM2.5-8B-A1B is Liquid AI's latest edge model — a hybrid convolution + attention MoE architecture with:
8.3B total parameters, 1.5B active per token (MoE with 32 experts, 4 active)
128K context window (up from 32K in LFM2)
Trained on 38T tokens with large-scale reinforcement learning
Reasoning model — generates <think>...</think> chains before answering
Fastest in its class: 18,500 tokens/sec on H100 at high concurrency
The architecture is not a standard Transformer. It combines:
18 layers of gated short convolutions (LIV blocks) — O(n) complexity
6 layers of Grouped Query Attention (GQA) — O(n²) for global context
MoE feed-forward with sparse expert routing
This hybrid design is what makes it fast. It's also what makes abliteration non-trivial.
Why standard abliteration tools don't work here
Every existing abliteration tool — NousResearch, Heretic, OBLITERATUS — targets standard Transformer weight matrices:
self_attn.o_proj ← doesn't exist in LFM2.5
mlp.down_proj ← doesn't exist in LFM2.5
Running sharded_ablate.py on LFM2.5 without patching results in 0 shards modified. The model is completely unchanged. This is why no abliterated version existed before this release.
How this was done
1. Architecture reverse engineering
Full manual inspection of the LFM2.5 weight map to identify the correct abliteration targets:
Key insight: conv.in_proj has shape [6144, 2048] — a 3x expansion projection that cannot be abliterated with the standard direction subtraction without a dimension mismatch error. Excluded intentionally.
2. Patch to sharded_ablate.py
python
1# LFM2/LFM2.5 hybrid architecture support patch2# by Gastón Parravicini — May 29, 20263# Enables abliteration of lfm2moe models in NousResearch/llm-abliteration45lfm2_patterns =[6f"{layer_prefix}.layers.{layer}.self_attn.out_proj.weight",7f"{layer_prefix}.layers.{layer}.conv.out_proj.weight",8f"{layer_prefix}.layers.{layer}.feed_forward.w2.weight",9]
Without this patch: 0/10 shards modified.
With this patch: 6/10 shards modified, all correct targets.
3. Refusal direction analysis
Used analyze.py to map refusal signal strength across all 24 layers:
Layers
Est. Signal Quality
Type
0–2
~0.000
Skip
3–10
0.010–0.062
Low
11–17
0.108–0.240
Peak — abliterated here
18–23
0.049–0.145
High
Layer 16 was the peak signal layer (Est. Signal Quality: 0.242). Used as the primary measurement reference for all ablated layers.
Tool calling: LFM2.5 supports tool calling natively in transformers. In llama.cpp there is a known bug with the chat template that breaks tool use — upstream is debugging (PR #23826).
Prompt cache: lfm2moe models clear the KV cache on every turn in llama.cpp (known upstream issue). Output quality is unaffected.
Reasoning: This is a thinking model. Responses include <think>...</think> before the final answer. This is expected and correct.
