This repository contains GGUF format quantizations of the model LiquidAI/LFM2.5-8B-A1B. These models were generated using llama.cpp with an importance matrix (imatrix) - https://gist.github.com/bartowski1182/82ae9b520227f57d79ba04add13d0d0d

Used this pull request to be able to quantize: https://github.com/ggml-org/llama.cpp/pull/23826/ Should work with the latest llama.cpp

Quantization below 4 bits may result in unacceptably low quality.