Views
No views yet

1import torch
2from transformers import pipeline
3
4pipe = pipeline(
5 "text-generation",
6 model="lemuralabs/Lemura Labs-Limitless-Gemma-2B",
7 model_kwargs={"torch_dtype": torch.bfloat16},
8 device="cuda", # replace with "mps" to run on a Mac device
9)
10
11messages = [
12 {"role": "user", "content": "who are you"},
13]
14
15outputs = pipe(messages, max_new_tokens=256)
16assistant_response = outputs[0]["generated_text"][-1]["content"].strip()
17print(assistant_response)| Model Version | Description |
|---|---|
| Lemura Labs-Limitless-Gemma-2B-Q2_K.gguf | Optimized for minimal memory usage with lower precision. Suitable for resource-constrained environments. |
| Lemura Labs-Limitless-Gemma-2B-Q4_K_M.gguf | Balances performance and precision, offering faster inference with moderate memory usage. |
| Lemura Labs-Limitless-Gemma-2B-Q8_0.gguf | Provides higher precision with increased memory usage, suitable for tasks requiring more accuracy. |
| Lemura Labs-Limitless-Gemma-2B-F16.gguf | Full 16-bit floating point precision for maximum accuracy, ideal for high-end GPUs. |