Views
No views yet

| Your GPU | Recommended quant | Weights |
|---|---|---|
| RTX 3090 / 4090 / 5090 (24 GB) | Q8_0 | ~2.9 GB |
| RTX 4080 / 5080 / 4060 Ti 16G (16 GB) | Q6_K | ~2.3 GB |
| RTX 3060 / 4070 / 5070 (12 GB) | Q5_K_M | ~2.0 GB |
| RTX 4060 / 3070 (8 GB) | Q4_K_M | ~1.8 GB |
| GTX 1660 Super / 2060 / 3050 laptop (6 GB) | IQ4_XS | ~1.6 GB |
| CPU-only / Apple Silicon | Q4_K_M | fits in system RAM |
| File | Format | Size |
|---|---|---|
lfm2.5-2.6b-fable5-coding-agent-heretic-F16.gguf | GGUF F16 | 5.03 GB |
lfm2.5-2.6b-fable5-coding-agent-heretic-Q2_K.gguf | GGUF Q2_K | 1.02 GB |
lfm2.5-2.6b-fable5-coding-agent-heretic-IQ3_S.gguf | GGUF IQ3_S | 1.18 GB |
lfm2.5-2.6b-fable5-coding-agent-heretic-Q3_K_S.gguf | GGUF Q3_K_S | 1.18 GB |
lfm2.5-2.6b-fable5-coding-agent-heretic-Q3_K_M.gguf | GGUF Q3_K_M | 1.27 GB |
lfm2.5-2.6b-fable5-coding-agent-heretic-Q3_K_L.gguf | GGUF Q3_K_L | 1.35 GB |
lfm2.5-2.6b-fable5-coding-agent-heretic-IQ4_XS.gguf | GGUF IQ4_XS | 1.42 GB |
lfm2.5-2.6b-fable5-coding-agent-heretic-Q4_K_S.gguf | GGUF Q4_K_S | 1.49 GB |
lfm2.5-2.6b-fable5-coding-agent-heretic-Q4_0.gguf | GGUF Q4_0 | 1.48 GB |
lfm2.5-2.6b-fable5-coding-agent-heretic-Q4_1.gguf | GGUF Q4_1 | 1.63 GB |
lfm2.5-2.6b-fable5-coding-agent-heretic-Q4_K_M.gguf | GGUF Q4_K_M | 1.56 GB |
lfm2.5-2.6b-fable5-coding-agent-heretic-Q5_K_S.gguf | GGUF Q5_K_S | 1.77 GB |
lfm2.5-2.6b-fable5-coding-agent-heretic-Q5_K_M.gguf | GGUF Q5_K_M | 1.81 GB |
lfm2.5-2.6b-fable5-coding-agent-heretic-Q6_K.gguf | GGUF Q6_K | 2.07 GB |
lfm2.5-2.6b-fable5-coding-agent-heretic-Q8_0.gguf | GGUF Q8_0 | 2.68 GB |
lfm2).llama serve -hf saidutta69/lfm2.5-2.6b-fable5-coding-agent-heretic to pull the default quant.1# defaults to the Q4_K_M quant
2llama serve -hf saidutta69/lfm2.5-2.6b-fable5-coding-agent-heretic:Q4_K_Mollama run hf.co/saidutta69/lfm2.5-2.6b-fable5-coding-agent-heretic:Q4_K_M1import torch
2from transformers import AutoModelForCausalLM, AutoTokenizer
3
4MODEL_ID = "saidutta69/lfm2.5-2.6b-fable5-coding-agent-heretic"
5
6tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
7model = AutoModelForCausalLM.from_pretrained(
8 MODEL_ID,
9 dtype=torch.bfloat16,
10 device_map="auto",
11)
12
13messages = [
14 {"role": "system", "content": "You are a helpful coding assistant."},
15 {"role": "user", "content": "Write a Python function that merges overlapping intervals."},
16]
17
18inputs = tokenizer.apply_chat_template(
19 messages, tokenize=True, add_generation_prompt=True, return_tensors="pt"
20).to(model.device)
21
22with torch.inference_mode():
23 output = model.generate(**inputs, max_new_tokens=512, temperature=0.1)
24
25print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))Made with ❤️ by RACER IS OP — follow for more uncensored models