You can deploy this model and start earning money today!

hlhsiao/
llama-3.2-3b-instruct-quantized

Name: llama-3.2-3b-instruct-quantized
Availability: InStock

Views

No views yet

LLaMA 3.2 3B Instruct - Quantized (w8a8)

This is a 3B parameter LLaMA 3.2 Instruct model, quantized with 8-bit weights and activations (w8a8). Model files include safetensors-formatted weights and generation_config.json.

Files

model-00001-of-00002.safetensors + index.json: quantized model weights
config.json + generation_config.json: model and generation configuration
recipe.yaml: quantization configuration (if applicable)

How to load

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("your-username/llama-3.2-3b-instruct-quantized", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("your-username/llama-3.2-3b-instruct-quantized")

hlhsiao/llama-3.2-3b-instruct-quantized

LLaMA 3.2 3B Instruct - Quantized (w8a8)

Files

How to load

hlhsiao/
llama-3.2-3b-instruct-quantized