Llama-Guard-3-1B-GGUF-INT8 (Optimized for Android)
This repository provides a quantized INT8 (Q8_0) version of Meta's Llama-Guard-3-1B, converted to the GGUF format for efficient on-device inference.
Built with Llama
Llama Guard 3-1B is a fine-tuned Llama-3.2-1B pretrained model for content safety classification. This specific GGUF version is a derivative work optimized for mobile and edge deployments where accuracy and memory efficiency are critical.
Why INT8 (Q8_0) instead of INT4?
While Meta provides an official 4-bit version, smaller models (1B) can experience a significant increase in False Positive Rates (FPR) when heavily quantized. For example, in German, the INT4 version's FPR can rise to ~14.5%, leading to excessive over-blocking.
This INT8 version maintains approximately 99% of the original model's precision, making it the "Sweet Spot" for developers who need reliable safety layers on Android without frustrating users through false alarms.
Model Details
- Base Model: meta-llama/Llama-Guard-3-1B
- Quantization: INT8 (Q8_0) via
llama.cpp
- File Size: ~1.6 GB
- Intended Use: Input/Output moderation for LLM-based applications.
Hazard Taxonomy & Policy
The model is trained to detect 13 categories of hazards based on the MLCommons taxonomy:
| Category | Description |
|---|
| S1 | Violent Crimes |
| S2 | Non-Violent Crimes |
| S3 | Sex-Related Crimes |
| S4 | Child Sexual Exploitation |
| S5 | Defamation |
| S6 | Specialized Advice (Medical/Legal) |
| S7 | Privacy Violations |
| S8 | Intellectual Property |
| S9 | Indiscriminate Weapons |
| S10 | Hate Speech |
| S11 | Suicide & Self-Harm |
| S12 | Sexual Content (Erotica) |
| S13 | Elections (Misinformation) |
Usage
Prompt Template
To ensure correct classification, you must use the official Llama-Guard template:
1<|begin_of_text|><|start_header_id|>user<|end_header_id|>
2
3[YOUR_INPUT_HERE]<|eot_id|><|start_header_id|>assistant<|end_header_id|>
License & Terms
This model is licensed under the Llama 3.2 Community License Agreement.
Attribution
"Llama 3.2 is licensed under the Llama 3.2 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved."
Acceptable Use Policy
You must comply with Meta's Acceptable Use Policy. By using this model, you agree not to use it for illegal activities, harassment, or any purposes prohibited by Meta's guidelines.
Citation
If you use this model, please cite the original Llama 3 family:
1@misc{metallamaguard3,
2 author = {Llama Team, AI @ Meta},
3 title = {The Llama 3 Family of Models},
4 howpublished = {\url{[https://github.com/meta-llama/PurpleLlama/blob/main/Llama-Guard3/1B/MODEL_CARD.md](https://github.com/meta-llama/PurpleLlama/blob/main/Llama-Guard3/1B/MODEL_CARD.md)}},
5 year = {2024}
6}
Quantization performed by: [Thorge Mrowinski]
Format: GGUF (Compatible with llama-android, llama.rn, and llama.cpp)
