[!WARNING] Research preview. It performs the style of reasoning, but the content is unreliable. This ~8B-ish model (aggressively pruned + LoRA-distilled) produces convincing chain-of-thought — <think> blocks, "let me think carefully", "but wait", multi-step hedging — but it routinely confabulates: it hallucinates numbers, misreads problems, and reaches wrong answers while sounding careful. Do not trust its outputs. Not evaluated for coherence, factuality, or alignment/safety. For research and curiosity only. Unofficial; not affiliated with Thinking Machines or NVIDIA.

A demonstration that reasoning style can be distilled into a tiny, heavily-pruned MoE — while reasoning ability cannot. Runs on a 16 GB Apple-silicon Mac.

What it is

Built from thinkingmachines/Inkling-Small (a 256-routed-expert MoE) by:

Pruning to 8 of 256 routed experts (~97% of routed params removed), 4-bit MLX.
LoRA-healing back to coherence on FineWeb-Edu → Wikipedia.
Instruction-tuning on OpenHermes-2.5.
Reasoning-distilling on chain-of-thought traces from ianncity/KIMI-K2.5-1000000x and nvidia/Nemotron-Cascade-2-SFT-Data.

The result talks like a careful reasoner. It cannot reliably be one — 8 experts is far too little capacity to restore the base model's knowledge or arithmetic. That gap is the whole point of the demo.

Example (rep-penalty 1.3, `User:/Assistant:` format)

User: A store has 15 apples and sells 8. How many are left?
Assistant: <think>
There is a simple arithmetic problem to find the number of remaining apples.
The store currently holds **15** apples. It plans to sell **8** apples from its inventory.
So we need to calculate: ...

Reads the problem faithfully, sets up the reasoning — then the actual computation falls apart. Representative: fluent reasoning form, unreliable substance.

How to run

GGUF (recommended) — needs the Inkling llama.cpp branch

Stock llama.cpp cannot load Inkling. Build PR #25731 (danielhanchen/llama.cpp, branch add-inkling), then:

bash

# CPU only: Metal offload is buggy in the PR build; -ngl 0 works (~14 tok/s)
llama-cli -m inkling-tiny-reasoning-v1-Q4_K_M.gguf -no-cnv -st -ngl 0 --temp 0 \
  -p $'User: If a train travels 60 mph for 2 hours, how far does it go?\n\nAssistant:'

Files: -Q4_K_M.gguf (~8 GB, fits 16 GB) and -Q8_0.gguf (~14 GB).
Use a repetition penalty (~1.3) — greedy loops on <think>. (Opposite of a concise chat model.)

MLX (adapter)

adapter/adapters.safetensors is the reasoning LoRA (rank 16, scale 2). Apply it over the 4-bit MLX base inkling-mlx-k8 (from cebopp1/inkling-tiny-preview) using the bundled inkling_mlx/ runtime + run.py. Requires mlx mlx-lm transformers numpy.

Limitations

Confabulates. Wrong numbers, misread problems, invented facts — stated confidently.
Not factual, not safety-tested, English-only, text-only. No red-teaming or refusal training.
Format-sensitive: best in User:/Assistant:, needs the rep penalty above.

Licenses & attribution

This is a modified derivative work; components carry different licenses. Your use must comply with all:

Base model — thinkingmachines/Inkling-Small: Apache-2.0.
MLX runtime (inkling_mlx/) — from mlx-community/Inkling-Small-mlx: Apache-2.0 (one-line moe.py stop_gradient change for trainability).
Reasoning data — ianncity/KIMI-K2.5-1000000x: Apache-2.0.
Reasoning data — nvidia/Nemotron-Cascade-2-SFT-Data: NVIDIA Open Model License — because this model was trained partly on that data, the NVIDIA Open Model License terms attach to it (hence this repo's license: other).

Unofficial. Not affiliated with, sponsored by, or endorsed by Thinking Machines or NVIDIA.

cebopp1/inkling-tiny-reasoning-v1

inkling-tiny-reasoning-v1