LlamaLite-1B-Q8 is a quantized (8-bit) version of the Meta Llama 3.2-1B-Instruct model, optimized for efficient inference on edge devices and resource-constrained environments. This model maintains high accuracy while significantly reducing memory footprint.
Model Details
Base Model: Meta Llama 3.2-1B-Instruct
Quantization: 8-bit (GGUF format)
Size: 1.31 GB
Framework: Llama.cpp
Optimized for: Offline use, low-power devices
Usage
This model is suitable for real-time applications such as:
Offline AI assistants
Embedded systems
Edge AI devices
Low-latency inference
Example Usage in Llama.cpp
./main -m LlamaLite-1B-Q8.gguf -p "Tell me about quantum computing"
