You can deploy this model and start earning money today!

shuff57/
lfm2-24b-phase1-reasoning-GGUF

Name: lfm2-24b-phase1-reasoning-GGUF
Availability: InStock

gguf

endpoints_compatible

conversational

Views

No views yet

LFM2-24B-A2B Phase 1 Reasoning — GGUF Q4_K_M GGUF quantization of shuff57/lfm2-24b-phase1-reasoning (https://huggingface.co/shuff57/lfm2-24b-phase1-reasoning). Model Description This is the Phase 1 reasoning fine-tuned version of LiquidAI's LFM2-24B-A2B (24B MoE, 2.3B active parameters) model, trained on 13,201 synthetic reasoning examples generated for the O.G.R.E. (OllamaGradingRubricEvaluator) project. The base model (LiquidAI/LFM2-24B-A2B) was further fine-tuned using Unsloth's QLoRA approach with:

LoRA rank: 16
LoRA alpha: 16
Target modules: ["q_proj", "k_proj", "v_proj", "out_proj", "in_proj", "w1", "w2", "w3"]
Sequence length: 8192
Packing: False (critical for LFM2 MoE routing stability)
Training on responses only (using unsloth.chat_templates.train_on_responses_only)
Optimizer: adamw_8bit
Learning rate: 2e-5
Epochs: 1 (effective steps: ~1,568)
Precision: bfloat16 After training, the LoRA adapters were merged into the base model and saved in 16-bit format. GGUF Quantization The merged model was converted to GGUF format for efficient CPU usage with llama.cpp and Ollama. Conversion Process

Model Loading: Loaded with device_map="cpu" to avoid MoE-related VRAM overflow during transformation.
HF → GGUF Conversion: Used the llama.cpp conversion pipeline:
- Cloned llama.cpp (commit post-Oct 2025, includes native lfm2_moe support)
- Configured with cmake -B llama.cpp/build -S llama.cpp -DCMAKE_BUILD_TYPE=Release
- Built all targets (cmake --build llama.cpp/build --config Release)
- Located the llama-quantize binary via glob search
Quantization: Applied Q4_K_M quantization: ./llama.cpp/build/bin/llama-quantize lfm2-24b-phase1-F16.gguf lfm2-24b-phase1-Q4_K_M.gguf Q4_K_M 4. Validation: The resulting GGUF file was tested with Ollama and llama.cpp for correct LFM2 MoE handling. Quantization Details

Format: GGUF (GPT-Generated Unified Format)
Quantization Method: Q4_K_M (4-bit with mixed precision, state-of-the-art balance)
Estimated Size: ~14.42 GB
VRAM Usage: Can be run on systems with <8GB VRAM when offloaded, or CPU-only
Context Length: 8192 tokens (matches training) Usage with Ollama Create a Modelfile: FROM ./lfm2-24b-phase1-Q4_K_M.gguf SYSTEM "You are a careful analytical reasoner. Think step by step before answering." RENDERER lfm2 PARSER lfm2 PARAMETER temperature 0.1 PARAMETER top_k 50 PARAMETER top_p 0.95 PARAMETER repeat_penalty 1.05 PARAMETER num_ctx 8192 Then: ollama create lfm2-24b-phase1-reasoning -f Modelfile ollama run lfm2-24b-phase1-reasoning "Your prompt here" Usage with llama.cpp llama-cli -m lfm2-24b-phase1-Q4_K_M.gguf -p "You are a careful analytical reasoner." --temp 0.1 -n 512 Training Data The Phase 1 reasoning model was trained on a synthetic dataset of 13,201 examples designed to teach step-by-step reasoning, logical deduction, and mathematical problem-solving. The dataset is available at:
shuff57/ogre-phase1-synth (https://huggingface.co/datasets/shuff57/ogre-phase1-synth) Related Models
Base Model: LiquidAI/LFM2-24B-A2B (https://huggingface.co/LiquidAI/LFM2-24B-A2B)
Phase 1 Reasoning (HF): shuff57/lfm2-24b-phase1-reasoning (https://huggingface.co/shuff57/lfm2-24b-phase1-reasoning) (16-bit merged weights)
Phase 2 Stat Grader: shuff57/lfm2-24b-grader (https://huggingface.co/shuff57/lfm2-24b-grader) (in progress) Citation If you use this model, please cite the O.G.R.E. project: @misc{ogre2026, title={O.G.R.E.: OllamaGradingRubricEvaluator}, author={shuff57}, year={2026}, publisher={Hugging Face}, url={https://huggingface.co/shuff57} } License Apache 2.0 — same as the base LFM2-24B-A2B model.

shuff57/lfm2-24b-phase1-reasoning-GGUF

shuff57/
lfm2-24b-phase1-reasoning-GGUF