Views
No views yet
| This model | FP16 baseline | |
|---|---|---|
| Decode tok/s (steady-state) | 138.33 | 45.34 |
| Prefill tok/s (steady-state) | 498.55 | 341.91 |
| Decode tok/s (avg, long traces) | 121.16 | 44.08 |
| Peak memory (GB) | 2.623 | 6.936 |
| Disk size (MB) | 1575 | 5902 |
Warmed, short-prompt, chat-templated, thinking disabled. Represents steady-state decode for typical chat use; long thinking traces will be slower due to KV-cache growth.
| Benchmark | This model | FP16 baseline | n |
|---|---|---|---|
| MATH-500 (math reasoning) | 76.7% (answered 24/30) | 93.3% (answered 30/30) | 30 |
| AIME 2024 (competition math) | 53.3% | 76.7% | 30 |
| IFEval (instruction following) | 54.5% | 61.4% | 44 |
| HumanEval (code, pass@1) | 86.7% | 73.3% | 30 |
| MMLU (knowledge, accuracy) | 42.0% | 40.0% | 50 |
| Level | This model | FP16 baseline |
|---|---|---|
| level 1 | 100.0% | 100.0% |
| level 2 | 100.0% | 100.0% |
| level 3 | 83.3% | 83.3% |
| level 4 | 66.7% | 100.0% |
| level 5 | 33.3% | 83.3% |
| Context length | Decode tok/s |
|---|---|
| ~128 tokens | 131.4 |
| ~256 tokens | 130.7 |
| ~512 tokens | 129.5 |
| ~1024 tokens | 128.2 |
<think>…</think> block and then writes its final answer, so the way you call it
matters:tokenizer.apply_chat_template(..., add_generation_prompt=True)).
Passing a raw string skips the <|im_start|> / <|im_end|> markers the model was trained on.max_tokens of at least 8192 (16384+ for hard competition math). Small caps cut it off mid-thought.\boxed{} — that's how it was trained and how the
benchmarks below were scored. Parse the text after </think> for the answer.eos_token_id to include <|im_end|> (151645) so
generation halts cleanly at the turn boundary.temperature=1.0, top_p=0.95, top_k disabled.The benchmark numbers above were produced with greedy decoding (temperature 0) for reproducibility — so a variant's score reflects quantization damage, not sampling noise. For everyday use, the recommended sampling settings give better, more diverse reasoning.
pip install mlx-lm1from mlx_lm import load, generate
2from mlx_lm.sample_utils import make_sampler
3
4model, tokenizer = load("sahilchachra/vibethinker-3b-mxfp4-mlx")
5
6messages = [{"role": "user",
7 "content": "Let x be the number of ways to ... . Give the final answer in \\boxed{}."}]
8prompt = tokenizer.apply_chat_template(
9 messages, add_generation_prompt=True, tokenize=False,
10)
11
12# Reasoning needs a large token budget; sampling per the model card.
13sampler = make_sampler(temp=1.0, top_p=0.95)
14response = generate(
15 model, tokenizer, prompt=prompt,
16 max_tokens=16384, sampler=sampler, verbose=True,
17)
18# `response` contains a <think>...</think> trace followed by the final answer.| Model | Variant |
|---|---|
| sahilchachra/vibethinker-3b-mxfp4-mlx | Block float MX FP4 ← this model |
| sahilchachra/vibethinker-3b-mxfp8-mlx | Block float MX FP8 |
| sahilchachra/vibethinker-3b-optiq-5bpw-mlx | OptiQ mixed-precision (target 5.0 bpw) |