AdvRahul/Axion-Lite-1B-Q5_K_M-GGUF
Axion-Lite-1B is a safety-enhanced, quantized version of Google's powerful gemma-3-1b-it model. This model has been specifically fine-tuned to improve its safety alignment, making it more robust and reliable for a wide range of applications.
The model is provided in the GGUF format, which allows it to run efficiently on CPUs and other hardware with limited resources.
🚀 Model Details
- Model Creator: AdvRahul
- Base Model: google/gemma-3-1b-it
- Fine-tuning Focus: Enhanced Safety & Harmlessness through red-teaming.
- Quantization:
Q5_K_M via GGUF. This quantization offers an excellent balance between model size, inference speed, and performance preservation.
- Architecture: Gemma 3
- License: Gemma 3 Terms of Use.
💻 How to Use
This model is in GGUF format and is designed to be used with frameworks like llama.cpp and its Python bindings.
Using llama-cpp-python
First, install the necessary library. Ensure you have a version that supports Gemma 3 models.
1
2pip install llama-cpp-python
Then, you can use the following Python script to run the model:
1from llama_cpp import Llama
2
3# Download the model from the Hugging Face Hub before running this
4# Or let llama-cpp-python download it for you
5llm = Llama.from_pretrained(
6 repo_id="AdvRahul/Axion-Lite-1B-Q5_K_M-GGUF",
7 filename="Axion-Lite-1B-Q5_K_M.gguf",
8 verbose=False
9)
10
11prompt = "What are the key principles of responsible AI development?"
12
13# The Gemma 3 instruction-tuned model uses a specific chat template.
14# For simple prompts, you can start with <start_of_turn>user\n{prompt}<end_of_turn>\n<start_of_turn>model
15chat_prompt = f"<start_of_turn>user\n{prompt}<end_of_turn>\n<start_of_turn>model"
16
17output = llm(chat_prompt, max_tokens=256, stop=["<end_of_turn>"], echo=False)
18
19print(output['choices'][0]['text'])
Using llama.cpp (CLI)
You can also run this model directly from the command line after cloning and building the llama.cpp repository.
1# Clone and build llama.cpp
2git clone [https://github.com/ggerganov/llama.cpp](https://github.com/ggerganov/llama.cpp)
3cd llama.cpp
4make
5
6# Run inference
7./main -m /path/to/your/models/Axion-Lite-1B-Q5_K_M.gguf -p "<start_of_turn>user\nWhat is the capital of India?<end_of_turn>\n<start_of_turn>model" -n 128
📝 Model Description
Fine-Tuning for Safety
Axion-Lite-1B originates from google/gemma-3-1b-it. The primary goal of this project was to enhance the model's safety alignment. The base model underwent extensive red-team testing with advanced protocols to significantly reduce the likelihood of generating harmful, unethical, biased, or unsafe content. This makes Axion-Lite-1B a more suitable choice for applications that require a higher degree of content safety and reliability.
Quantization
The model is quantized to Q5_K_M, a method that provides a high-quality balance between perplexity (model accuracy) and file size. This makes it ideal for deployment in resource-constrained environments, such as on local machines, edge devices, or cost-effective cloud instances, without a significant drop in performance.
ℹ️ Base Model Information (Gemma 3)
<details>
<summary>Click to expand details on the base model</summary>
Gemma is a family of lightweight, state-of-the-art open models from Google, built from the same research and technology used to create the Gemini models. Gemma 3 models handle text input and generate text output, with open weights for both pre-trained variants and instruction-tuned variants. The 1B model was trained on 2 trillion tokens of data.
Training Data
The base model was trained on a dataset of text data that includes a wide variety of sources:
- Web Documents: A diverse collection of web text ensures the model is exposed to a broad range of linguistic styles, topics, and vocabulary in over 140 languages.
- Code: Exposing the model to code helps it learn the syntax and patterns of programming languages.
- Mathematics: Training on mathematical text helps the model learn logical reasoning and symbolic representation.
Data Preprocessing
The training data for the base model underwent rigorous cleaning and filtering, including:
- CSAM Filtering: Exclusion of Child Sexual Abuse Material.
- Sensitive Data Filtering: Automated techniques were used to filter out certain personal information and other sensitive data.
- Content Quality Filtering: Filtering based on content quality and safety in line with Google's policies.
</details>
⚠️ Ethical Considerations and Limitations
While this model has been fine-tuned to enhance its safety, no language model is perfectly safe. It inherits the limitations of its base model, gemma-3-1b-it, and the data it was trained on.
- Potential for Bias: The model may still generate content that reflects societal biases present in the training data.
- Factual Inaccuracy: The model can "hallucinate" or generate incorrect or outdated information. It should not be used as a sole source of truth.
- Not a Substitute for Human Judgment: The outputs should be reviewed and validated, especially in sensitive or high-stakes applications.
Developers implementing this model should build additional safety mitigations and content moderation tools as part of a defense-in-depth strategy, tailored to their specific use case.
Citing the Base Model
If you use this model, please consider citing the original Gemma 3 work:
1@article{gemma_2025,
2 title={Gemma 3},
3 url={[https://goo.gle/Gemma3Report](https://goo.gle/Gemma3Report)},
4 publisher={Kaggle},
5 author={Gemma Team},
6 year={2025}
7}
