qquark 435M v0.1
qquark 435M is a small local prompt-enhancer model for AI code agents.
It rewrites rough user requests into clearer agent-ready prompts before sending them to larger coding agents such as Codex, Roo, Cline, OpenCode, Claude Code, local Qwen/OmniCoder pipelines, or other agentic coding tools.
Status: early alpha / research prototype.
Why qquark exists
Large coding agents often perform better when the user request is clear, scoped, and explicit.
qquark 435M acts as a lightweight local preprocessing model:
1 rough user request
2 ↓
3 qquark 435M
4 ↓
5 agent-ready prompt
6 ↓
7 larger code agent
The goal is not to replace a coding model.
Example
Input:
create a simple browser gameOutput:
Create a simple browser game using HTML, CSS, and JavaScript in a single file. The game should feature a single-screen interface, clear input controls, and a restart button. Ensure the code is clean, commented, and ready to run immediately without external dependencies.Model
Field Value Name qquark 435M Version v0.1 Parameters ~435M Format GGUF Recommended quant Q4_K_M File size ~277 MB Runtime llama.cpp Purpose local prompt enhancement for AI code agents Status early alpha
Recommended inference settings
1 temperature: 0.15
2 top_k: 10
3 top_p: 0.85
4 min_p: 0
5 repeat_penalty: 1.18
6 max_tokens: 80–120
7 stop: <|im_end|>
Installation
1. Clone the repository
1 git clone https://github.com/TheRofli/qquark-435m.git
2 cd qquark-435m
2. Install Python dependencies
1 python -m venv venv
2 source venv/bin/activate
3 pip install -r requirements.txt
3. Install llama.cpp
Build llama.cpp with CUDA if you want GPU acceleration:
1 git clone https://github.com/ggml-org/llama.cpp ~/llm/llama.cpp
2 cd ~/llm/llama.cpp
3 cmake -B build -DGGML_CUDA = ON
4 cmake --build build --config Release -j
Add llama.cpp binaries to PATH, or use the full binary path.
Download the model
Download:
qquark-435m-v0.1-byte-Q4_K_M.ggufRun with llama.cpp
Start the server:
./scripts/run_llama_cpp.sh release/qquark-435m-v0.1-byte-Q4_K_M.ggufOr manually:
1 llama-server \
2 -m release/qquark-435m-v0.1-byte-Q4_K_M.gguf \
3 --host 127.0 .0.1 \
4 --port 8088 \
5 --ctx-size 2048 \
6 --n-gpu-layers 999 \
7 --flash-attn on
Use qquark CLI
In another terminal:
python -m qquark.cli "create a simple browser game"With automatic project context:
python -m qquark.cli --context /path/to/project "make the button round"Without context:
python -m qquark.cli --no-context "why is training killed"Custom llama.cpp server:
1 python -m qquark.cli \
2 --server http://127.0.0.1:8088 \
3 "make a task for the agent to improve the UI"
Context Builder
qquark works best when it receives project context.
The CLI includes a simple Context Builder that detects common project markers:
Marker Meaning package.jsonJS/TS project vite.config.tsVite frontend next.config.jsNext.js *.uprojectUnreal Engine CMakeLists.txtC/C++ pyproject.tomlPython Cargo.tomlRust __manifest__.pyOdoo docker-compose.ymlDocker Compose
Example:
python -m qquark.cli --context ~/my-unreal-game "кнопку круглой сделай"Internally, qquark receives a prompt with detected context and constraints, so it is less likely to suggest the wrong stack.
How to integrate into an app
Recommended app architecture:
1 User input
2 ↓
3 Context Builder
4 ↓
5 qquark 435M via llama.cpp
6 ↓
7 cleaned agent-ready prompt
8 ↓
9 larger coding agent
Important runtime behavior:
Use llama.cpp /completion, not chat mode, for v0.1.
Build the prompt manually using qquark's prompt template.
Stop on <|im_end|>.
Also post-process by cutting everything after <|im_end|>.
Use low temperature.
Minimal post-processing:
1 def clean_qquark_output ( text : str ) - > str :
2 text = text . split ( "<|im_end|>" , 1 ) [ 0 ]
3 text = text . split ( "<|im_start|>" , 1 ) [ 0 ]
4 return text . strip ( )
Limitations
qquark 435M v0.1 is an early prototype.
Known limitations:
Not a general assistant.
Not a coding model.
It may hallucinate technologies if no context is provided.
Git safety is not perfect.
It can sometimes repeat phrases.
It should be used with low temperature.
Outputs should be post-processed after <|im_end|>.
Context-aware behavior is experimental.
Training summary
Architecture: decoder-only Transformer
Parameters: ~435M
Vocabulary: 32k + byte fallback for llama.cpp GGUF compatibility
Trained from scratch
SFT on synthetic prompt-enhancement data
Masked SFT: loss applied only to assistant output
Exported to HF/LLaMA-compatible format
Converted to GGUF for llama.cpp runtime
Hardware used: RTX 4070 SUPER 12GB, WSL2
Roadmap
License
The code in this repository is released under the MIT License.
The qquark 435M model weights are released under the Apache-2.0 License.
See LICENSE for code licensing details.
