LFM2.5-2.6B is a compact general-purpose text model developed by Liquid AI as part of the LFM2.5 family of hybrid models designed for efficient on-device deployment. The model combines a lightweight architecture with agentic post-training, a 128K context window, and capabilities for tool use, instruction following, multi-step agentic workflows, data extraction, and retrieval-augmented generation (RAG).

This repository contains GGUF quantized variants of LFM2.5-2.6B optimized for efficient local inference using llama.cpp. The quantized formats substantially reduce the model's memory footprint while retaining its core language understanding and agentic capabilities.

LFM2.5-2.6B is a text-only model, not a vision-language or multimodal model. Its primary focus is efficient agentic language processing, particularly for applications where low latency, long context, tool interaction, and local execution are important.

Model Overview

Model Name: LFM2.5-2.6B
Base Model: LiquidAI/LFM2.5-2.6B
Architecture: Hybrid LFM2 architecture with short-convolution blocks and grouped-query attention
Parameter Count: 2.69 Billion Parameters
Modalities: Text
Context Length: 131,072 tokens
Vocabulary Size: 128,000
Languages: Multilingual
Developer: Liquid AI
License: LFM1.0

Quantization Formats

This repository provides GGUF quantized versions of LFM2.5-2.6B optimized for efficient local inference using llama.cpp.

IQ3_M

Size reduction of approx 76.14% (1.20 GB) compared to 16-bit (5.03 GB)
Aggressive 3-bit quantization designed for highly memory-efficient deployment of the 2.6B model
Suitable for edge AI, local agent workloads, and systems where available memory is limited
Enables practical execution of long-context language workloads with a substantially smaller model footprint
More demanding reasoning, tool-use sequences, and complex multi-step interactions may experience greater quality degradation than higher-precision variants

IQ4_NL

Size reduction of approx 70.38% (1.49 GB) compared to 16-bit (5.03 GB)
Advanced 4-bit non-linear quantization providing a stronger balance between model fidelity and memory reduction
Well suited for agentic workflows, tool interaction, RAG, structured information extraction, and long-context processing
Provides improved retention of language and reasoning behavior compared with more aggressive 3-bit compression
May require additional computational resources compared with the smaller IQ3_M variant

IQ4_XS

Size reduction of approx 71.77% (1.42 GB) compared to 16-bit (5.03 GB)
Compact 4-bit quantization offering a balanced deployment profile for local and edge inference
Suitable for conversational workloads, multi-step agent tasks, retrieval-based applications, and structured data extraction
Provides a practical compromise between model footprint and generation quality
Appropriate for deployments where memory efficiency and stable agentic behavior are both important

Training Background (Original Model)

LFM2.5-2.6B is built on the LFM2 hybrid architecture and was pretrained on approximately 34 trillion tokens. Its mid-training process extended the model's context capability to 128K tokens. The post-training pipeline subsequently transformed the base model into an agent-oriented model.

Pretraining

Approximately 34T training tokens used during pretraining
Hybrid architecture combining double-gated short-convolution blocks with grouped-query attention
Mid-training stage used to extend context capability to 128K tokens
Designed for efficient language processing and deployment across constrained hardware environments

Agentic Post-Training

The post-training process consists of multiple stages:

Supervised fine-tuning performed in two rounds
Per-domain teacher specialization
Multi-domain on-policy distillation
Agentic reinforcement learning using popular agentic harnesses

The agentic reinforcement-learning stage exposes the model to tools, system prompts, and interaction patterns used by agent environments, improving its ability to operate through multi-step tool-based workflows.

Key Capabilities

Agentic Task Execution Designed to work through multi-step tasks rather than limiting interaction to single-turn text generation.
Tool Use & Function Calling Supports tool definitions, function calls, tool-result interpretation, and final response generation within an agent workflow.
Long-Context Processing Supports a context length of up to 131,072 tokens, making it suitable for applications involving large inputs and extended interactions.
Data Extraction Suitable for extracting structured information from unstructured textual inputs.
Retrieval-Augmented Generation Recommended by Liquid AI for RAG-oriented applications where retrieved information must be incorporated into multi-step language workflows.
Instruction Following Post-training emphasizes reliable adherence to instructions across conversational and agentic workflows.
Multilingual Processing Supports 16 languages spanning European, Asian, and other language groups.
Efficient Local Inference The LFM2 architecture is specifically designed for efficient deployment across CPU, GPU, and other constrained hardware environments. Liquid AI reports 220 tokens/s on an Apple M5 Max and 113 tokens/s on an AMD Ryzen AI Max+ 395 for the original model configuration.

Recommended Usecases

Agentic AI Systems Build autonomous or semi-autonomous systems capable of performing multi-step tasks.
Tool-Calling Applications Integrate the model with external functions, APIs, databases, and application tools.
RAG Systems Build retrieval-augmented assistants capable of processing retrieved knowledge within long contexts.
Structured Data Extraction Extract organized information from large volumes of unstructured text.
Long-Context Applications Process large documents and extended conversational contexts within a 128K-token context window.
On-Device AI Deploy language intelligence locally on phones, laptops, edge systems, and other constrained devices.
Conversational Assistants Build lightweight assistants requiring low-latency local inference.

Limitations

LFM2.5-2.6B is primarily optimized for agentic workloads, tool use, data extraction, RAG, and long-context applications. The official model card specifically states that it is not recommended for agentic coding or knowledge-heavy tasks.

Usage Example

Using llama.cpp

bash

./llama-cli \
  -m SandLogicTechnologies/LFM2.5-2.6B-IQ4_NL.gguf \
  -p "Analyze the provided information, identify the important entities, and return the results as structured JSON."

Acknowledgments

These quantized models are based on the original work by the Liquid AI development team.

Special thanks to:

The Liquid AI team for developing and releasing LFM2.5-2.6B.
The developers and open-source community behind llama.cpp for enabling efficient GGUF inference across CPU, GPU, and edge hardware.

Contact

For questions, feedback, or support, please reach out at support@sandlogic.com or visit https://www.sandlogic.com/

SandLogicTechnologies/LFM2.5-2.6B-GGUF

LFM2.5-2.6B