GGUF quantizations of TIM-360M-instruct for llama.cpp, Ollama, and LM Studio. This is the instruction-tuned (SFT+DPO) chat model — the one to use for conversation.

Arch qwen3, byte-level BPE (Llama-3-family pre-tokenizer), ChatML chat template, generation stops on <|im_end|>.

Files

File	Quant	Size	Use when
`tim-360m-instruct.F16.gguf`	F16	692 MB	Reference / lossless
`tim-360m-instruct.Q8_0.gguf`	Q8_0	369 MB	Near-lossless
`tim-360m-instruct.Q4_K_M.gguf`	Q4_K_M	258 MB	Smallest; best size/quality for CPU

Usage

Ollama:

ollama run hf.co/timothywong731/tim-360m-instruct-GGUF:Q4_K_M "Explain what a transformer is in two sentences."

llama.cpp (conversation mode applies the ChatML template automatically):

llama-cli -m tim-360m-instruct.Q4_K_M.gguf -cnv -p "You are a helpful assistant."

Limitations

360M parameters, ~30B pretraining tokens + light post-training. Basic instruction-following — short, sometimes shallow answers, weak arithmetic and multi-step reasoning, confident hallucination. English-only. No safety tuning — filter before exposing to end users.

Provenance & license

Apache-2.0. See the source model card for architecture, training data, and evaluation.