This repository contains weighted / imatrix GGUF quantizations of Firefly-v4. Imatrix quantization uses calibration data to guide compression, making it especially useful for low-bit quants where every bit has to earn its rent.

These files are intended for local inference with GGUF-compatible software such as:

Original model: https://huggingface.co/Guilherme34/Firefly-v4

Static GGUF repository: https://huggingface.co/mradermacher/Firefly-v4-GGUF

Sorted by file size, not necessarily quality. IQ quants are often preferable over similarly sized non-IQ quants.

Download one of the .gguf files and load it in your preferred GGUF-compatible backend.

Example with llama.cpp:

For multimodal usage, make sure your backend supports the model architecture and has the required projection files from the static GGUF repository when needed.

If you are new to GGUF files, you can also look at community GGUF README examples for setup notes, backend flags, and multi-part file handling.

Original Model: Firefly-v4
Author: Guilherme34

Original model page: https://huggingface.co/Guilherme34/Firefly-v4

Quantization repository: https://huggingface.co/mradermacher/Firefly-v4-i1-GGUF

This repository contains GGUF quantizations. All credit for the model training, fine-tuning, datasets, and original work belongs to the original author.

This repository follows the Firefly-v4 Attribution License.

Commercial use, quantizations, merges, and derivative works are permitted under the terms described in the included LICENSE file.

Please preserve the required attribution when redistributing or building upon this model.

This model is uncensored and may generate content without built-in refusals. It is intended for creative fiction and roleplay between consenting adults. The creator is not responsible for how the model is used. Do not use it to produce content that is illegal in your jurisdiction.