This repository contains GGUF quantizations of Atlas-Coder-0.5B — a coding-specialized language model fine-tuned from Qwen2.5-Coder-0.5B base using QLoRA on 80K decontaminated code instructions.
These GGUF files are optimized for CPU inference on consumer hardware using llama.cpp, Ollama, and LM Studio. No GPU required.
Highest quality — 8-bit, near-lossless. Use if RAM allows.
Atlas-Coder-0.5B-Q6_K.gguf
Q6_K
506 MB
Best quality/size balance — 6-bit K-quant. Recommended default.
Atlas-Coder-0.5B-Q5_K_M.gguf
Q5_K_M
420 MB
Great balance — 5-bit medium K-quant. Minimal quality loss.
Atlas-Coder-0.5B-Q4_K_M.gguf
Q4_K_M
398 MB
Most compressed — 4-bit medium K-quant. Best for low-RAM devices.
Which Quantization Should I Use?
Q8_0 → Maximum fidelity, RAM not a concern, you want the closest to FP16 output
Q6_K → Best all-rounder. This is the one to use if you are unsure
Q5_K_M → Excellent quality at a smaller footprint. Barely distinguishable from Q6_K for most tasks
Q4_K_M → Minimum RAM usage. Still surprisingly capable for a 400MB file
All four files run comfortably on any modern laptop with 4GB+ RAM.
Quick Start
Ollama
bash
1# Download the model file first, then create a Modelfile2cat> Modelfile <<EOF
3FROM ./Atlas-Coder-0.5B-Q5_K_M.gguf
45SYSTEM "You are Atlas-Coder, an elite AI coding assistant created by Siddharth N.R. You write clean, efficient, and well-documented Python code. You specialize in code generation, completion, debugging, refactoring, algorithm implementation, and software engineering reasoning."
67PARAMETER temperature 0.3
8PARAMETER top_p 0.9
9PARAMETER repeat_penalty 1.1
10PARAMETER num_predict 512
11EOF1213ollama create atlas-coder -f Modelfile
14ollama run atlas-coder "Write a Python function to reverse a linked list"
LM Studio
Open LM Studio
Go to Search tab
Search for Siddh07ETH/Atlas-Coder-0.5B-GGUF
Download your preferred quantization
Load and chat
llama.cpp (CLI)
bash
1# Clone and build llama.cpp2git clone https://github.com/ggerganov/llama.cpp
3cd llama.cpp &&make45# Run inference6./llama-cli \7 -m Atlas-Coder-0.5B-Q5_K_M.gguf \8 -p "<|im_start|>system\nYou are Atlas-Coder, an elite AI coding assistant.<|im_end|>\n<|im_start|>user\nWrite a Python binary search function.<|im_end|>\n<|im_start|>assistant\n"\9 -n 512\10 --temp 0.3\11 --top-p 0.9\12 --repeat-penalty 1.1\13 --no-display-prompt
Python (llama-cpp-python)
python
1from llama_cpp import Llama
23llm = Llama(4 model_path="Atlas-Coder-0.5B-Q5_K_M.gguf",5 n_ctx=1024,6 n_threads=4,7 verbose=False,8)910prompt ="""<|im_start|>system
11You are Atlas-Coder, an elite AI coding assistant created by Siddharth N.R. You write clean, efficient, and well-documented Python code.<|im_end|>
12<|im_start|>user
13Write a Python function to check if a number is prime.<|im_end|>
14<|im_start|>assistant
15"""1617output = llm(18 prompt,19 max_tokens=512,20 temperature=0.3,21 top_p=0.9,22 repeat_penalty=1.1,23 stop=["<|im_end|>","<|im_start|>"],24)2526print(output["choices"][0]["text"])
Recommended Generation Settings
Setting
Value
Reason
temperature
0.2–0.4
Conservative — reduces hallucinations in code
top_p
0.9
Focused vocabulary sampling
repeat_penalty
1.1
Prevents repetitive patterns
max_tokens
256–512
Sufficient for most coding tasks
ctx_size
1024
Matches training sequence length
Chat Template
Atlas-Coder uses ChatML format. Always wrap prompts in this structure:
<|im_start|>system
You are Atlas-Coder, an elite AI coding assistant created by Siddharth N.R. You write clean, efficient, and well-documented Python code. You specialize in code generation, completion, debugging, refactoring, algorithm implementation, and software engineering reasoning.<|im_end|>
<|im_start|>user
YOUR QUESTION HERE<|im_end|>
<|im_start|>assistant
Original Model
This GGUF repository is derived from the full FP16 model:
Atlas-Coder-0.5B is a coding-specialized LLM instruction-tuned from Qwen2.5-Coder-0.5Bbase (not instruct) using QLoRA on a Tesla T4. Key training highlights:
80K samples across 4 code datasets, including 50K execution-verified OSS-Instruct samples
n-gram Jaccard decontamination against HumanEval — benchmark scores are honest
Response-only loss masking — gradients only on assistant code output
Trained entirely on free Kaggle T4 GPU — reproducible on consumer hardware
