Views
No views yet

| Quantization | Bits per Weight | Ideal For | Link |
|---|---|---|---|
| Q8_0 | 8-bit | Best accuracy and performance | [model-Q8_0.gguf](https://huggingface.co/lemuralabs/Lemura Labs-Gemma-3-27B-Instruct-Uncensored-GGUF/blob/main/model-Q8_0.gguf) |
| Q6_K | 6-bit | Strong accuracy and fast inference | [model-Q6_K.gguf](https://huggingface.co/lemuralabs/Lemura Labs-Gemma-3-27B-Instruct-Uncensored-GGUF/resolve/main/model-Q6_K.gguf) |
| Q5_K_M | 5-bit | Balance between accuracy and speed | [model-Q5_K_M.gguf](https://huggingface.co/lemuralabs/Lemura Labs-Gemma-3-27B-Instruct-Uncensored-GGUF/resolve/main/model-Q5_K_M.gguf) |
| Q3_K_M | 3-bit | Low memory usage, good performance | [model-Q3_K_M.gguf](https://huggingface.co/lemuralabs/Lemura Labs-Gemma-3-27B-Instruct-Uncensored-GGUF/resolve/main/model-Q3_K_M.gguf) |
| TQ2_0 | 2-bit (Tiny) | Maximum speed and minimal resources | [model-TQ2_0.gguf](https://huggingface.co/lemuralabs/Lemura Labs-Gemma-3-27B-Instruct-Uncensored-GGUF/resolve/main/model-TQ2_0.gguf) |
| TQ1_0 | 1-bit (Tiny) | Minimal footprint and fastest inference | [model-TQ1_0.gguf](https://huggingface.co/lemuralabs/Lemura Labs-Gemma-3-27B-Instruct-Uncensored-GGUF/resolve/main/model-TQ1_0.gguf) |
Q8_0 or Q6_K.Q5_K_M.Q3_K_M, TQ2_0, or TQ1_0.1from transformers import AutoModelForCausalLM, AutoTokenizer
2import torch
3
4model_name = "lemuralabs/Lemura Labs-Gemma-3-27B-Instruct-Uncensored"
5
6tokenizer = AutoTokenizer.from_pretrained(model_name)
7model = AutoModelForCausalLM.from_pretrained(model_name)
8
9prompt = "Tell me a futuristic story about space travel."
10inputs = tokenizer(prompt, return_tensors="pt")
11output = model.generate(**inputs, max_length=200)
12print(tokenizer.decode(output[0], skip_special_tokens=True))