An 8B chat model whose every transformer linear is stored five-valued (sub-2 bits per weight) in a single 2.56 GB container, and which loads as a native transformers model in one from_pretrained call.

This is the ONE published 8B SKU. Cross-model rows we publish are same-harness measurements: the rival's checkpoint run on our protocol, with thinking disabled on both sides.

Container: neutrino-8b_v4.bin — TRTC v4, arch-3 (Qwen3 topology), 36 layers, hidden 4096, vocab 151936. 3,875,404,812 bytes, sha256 016c6f362ad237a8cf4043601efbc1e766574f76ea58691e78fdef51b4155fa0.
Inside: 252 packed trit linears + int8 embedding lanes (untied). Nothing in the decode path is fp16/fp32 weights; scales are snapped to the runtime grid at export.
Base: Qwen/Qwen3-8B (Apache-2.0), ternary QAT + staged post-training by Fermion Research. Tokenizer: Qwen3-8B (shipped in this repo).

Architecture

Geometry as read from the shipped container's header (bit-checked during the GGUF conversion, gguf/receipts/convert_8b.json; byte budget from the provenance-gated container walk):

field	value
Parameters	8,190,735,360 (6,945,767,424 ternary projection + 1,244,659,712 int8 embedding + 308,224 fp32 norm)
Decoder layers	36
Hidden width	4,096
Feed-forward width	12,288, gated (SwiGLU)
Attention	grouped-query 4:1 — 32 query heads, 8 KV heads, head_dim 128
Rotary embedding	full head width (rotary_dims 128), theta 1,000,000
Normalization	RMSNorm, eps 1e-6, plus per-head Q/K RMSNorm in attention
Context length	40,960 tokens (`max_pos`)
Vocabulary	151,936
Embeddings	untied — separate int8 input-embedding and lm_head records
KV cache	144 KiB/token fp16 (288 KiB fp32): 0.60 GB @ 4k, 4.83 GB @ 32k

Byte budget of the 3,875,404,812-byte container, by tensor class:

lane	bytes	share
ternary weight lane (q/k/v/o/gate/up/down × 36 layers = 252 linears)	2,604,662,784	67.2%
token embeddings, int8 (two untied 622 MB records)	1,244,659,712	32.1%
per-row metadata (dims, scales, row sums)	24,849,360	0.6%
norm vectors, fp32 (145 tensors)	1,232,896	0.03%
container header	60	—

Every layer costs exactly 72,351,744 weight-lane bytes (21.7% attention / 78.3% MLP). State occupancy across the 6.95B ternary weights: 62.63% zero / 18.68% plus / 18.69% minus.

Formats and artifacts

artifact	bytes	sha256
`neutrino-8b_v4.bin` (TRTC v4 container, the file every runtime executes)	3,875,404,812	`016c6f362ad237a8cf4043601efbc1e766574f76ea58691e78fdef51b4155fa0`
`neutrino-8b_v4.tv4z` (lossless coded transport, 66.05% of raw)	2,559,836,297	`83ec7a52bd6733e66fb546e89b9c6aa8b3411af7f648e9c0b2bb56ba50aac4df`
`gguf/neutrino-8b-fv5.gguf` (llama.cpp-fork pack, bit-checked vs the container)	4,093,015,136	`1c13a34360b2531d28821fb6aee41708f99a040215a2522af37213c6c8c87211`

The tv4z transport decodes back to the container byte-exactly (md5+sha round-trip identity; 17 s encode / 6 s decode on an M5). Three distribution surfaces read these artifacts:

pip engine (pip install fermion-research) — the one-command door; pulls the container and the platform-matching bin/ runtime from this repo. Measured: 24.94 tok/s CPU-only on an Apple M5, 9 threads.
GGUF pack + our llama.cpp fork (gguf/) — the llama.cpp-ecosystem door, CPU + CUDA. The pack uses our FV5 tensor type, so it loads through our fork only: build it once (two commands, below) and standard llama.cpp tooling works from there. Stock llama.cpp, ollama, and LM Studio builds do not include FV5 yet. Measured: 35.4 tok/s on an NVIDIA L4 at full offload (rebuilt batched kernel, 2026-07-30), 4.68 GiB VRAM @ 4k context (fits 8 GB cards).
MLX pack (mlx/) — Python-native Apple-silicon runtime with custom Metal kernels. Measured: 25.0 tok/s median on a 16 GB M5 under a 6 GiB memory cap.

Quickstart

bash

pip install fermion-research
printf 'In one sentence, why is the sky blue?\n/exit\n' | \
  fermion chat --model fermionresearch/Neutrino-8B --max-new 48

The first fermion chat downloads ~3.9 GB (container + tokenizer + native runtime) with no progress display in 0.1.5, so the terminal sits quiet for a few minutes on first run; later runs load from the local cache. (A progress display is coming in 0.1.6.)

Or pure transformers — download the repo first, because the 0.1.5 loader resolves the container relative to a local path only (passing the hub id straight to from_pretrained raises FileNotFoundError):

bash

hf download fermionresearch/Neutrino-8B --local-dir Neutrino-8B \
    --exclude "gguf/*" --exclude "*.tv4z"   # skip the packs other runtimes use

python

import fermion  # registers the trtc_v4 model type
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("Neutrino-8B")   # the downloaded directory
tokenizer = AutoTokenizer.from_pretrained("Neutrino-8B")
inputs = tokenizer.apply_chat_template([{"role": "user", "content": "hi"}],
                                       add_generation_prompt=True,
                                       enable_thinking=False,
                                       return_tensors="pt", return_dict=True)
out = model.generate(**inputs, max_new_tokens=256)  # generation_config: greedy
n = inputs["input_ids"].shape[1]
print(tokenizer.decode(out[0][n:], skip_special_tokens=True))

import fermion must come first: it is what registers the trtc_v4 model type, and without it AutoTokenizer/AutoModelForCausalLM cannot read this repo's config.json. return_dict=True is not optional either on transformers 4.56 and newer, where apply_chat_template returns a mapping rather than a bare tensor.

If you skip the import you will get this, and the advice in it is a dead end:

ValueError: The checkpoint you are trying to load has model type `trtc_v4`
but Transformers does not recognize this architecture. ... You can update
Transformers with the command `pip install --upgrade transformers`.

Upgrading Transformers will never fix it, and neither will installing from source: trtc_v4 is registered at import time by the fermion-research package, so no Transformers release knows it. trust_remote_code=True does not help either, because this repo carries no auto_map. Add import fermion above the Transformers import.

generation_config.json in this repo is deliberately greedy with no repetition penalty and no top-k: that is the setting the published GSM8K, IFEval and BFCL numbers were measured at, so a bare generate(), an lm-eval run and fermion generate all reproduce each other. The conversational and long-form settings are one argument away — see Recommended settings.

GGUF surface (build our public fork once — branch fermion-fv5 at fermionresearch/llama.cpp, = upstream ggml-org/llama.cpp @ d67c0b41 + the FV5 patch — then standard llama.cpp tooling; full instructions in gguf/README.md):

bash

hf download fermionresearch/Neutrino-8B gguf/neutrino-8b-fv5.gguf --local-dir .
git clone https://github.com/fermionresearch/llama.cpp && cd llama.cpp
git checkout fermion-fv5
cmake -B build -DCMAKE_BUILD_TYPE=Release -DLLAMA_CURL=OFF -DGGML_METAL=OFF  # no FV5 Metal kernel yet: CPU/CUDA only
cmake --build build -j --target llama-completion
./build/bin/llama-completion -m ../gguf/neutrino-8b-fv5.gguf \
    -p "Why is the sky blue?" -n 256 --temp 0 -no-cnv

MLX surface (Apple silicon — run from the mlx/ folder of the downloaded repo: the fermion_mlx package and its tokenizer files ship there, and requirements.txt installs mlx itself; full instructions in mlx/README.md):

bash

cd Neutrino-8B/mlx          # the directory downloaded above
pip install -r requirements.txt
python -m fermion_mlx --model ../neutrino-8b_v4.bin --mode chat \
    --tokenizer . --prompt "Why is the sky blue?"

The pip path is the reference torch path: correctness-gated (bit-packed weights at rest, 0 greedy mismatches vs the native runtime over 768 tokens; receipt retained internally), honest about speed — it is NOT the fast path. The fast path ships in this repo under bin/ (see below).

Verified end-to-end in one fresh CPU container (Modal, cpu=8, 24 GiB, no GPU): wheel install -> container sha gate -> piped chat produced a real answer, verdict PASS. Peak resident memory for load+chat was under 8 GiB (bit-packed weights, max_child_rss_gib 7.87 in the internal acceptance receipt).

Recommended settings, by use case

Swept on a predecessor container; the sampling sweep was NOT re-run for 016c6f36… across four prompt slices — chat, factual, structured/tool-shaped, long-form — at four temperature x repetition-penalty cells. — means leave it alone; a number there would be one we did not measure.

use case	temperature	top-p	top-k	repetition penalty	penalty window	max new tokens	stop
conversation / assistant	0.01 on the C binary · 0 on torch	1.0	not exposed	1.05	256	512	EOS `151645`
tool calling / structured output	0 (greedy)	—	—	1.0 (off)	—	256	EOS `151645`
long-form prose	0.7	0.95	not exposed	1.05	256	1024	EOS `151645`
deterministic / scriptable / benchmarks	0 (greedy)	—	—	1.0 (off)	—	256	EOS `151645`

Structured output is an 8B capability. On tool-shaped prompts this model emitted whole-reply, fence-free, parseable JSON 8/8 in every cell tried, including at temperature 0.7 — the format is not sensitive to the sampler here. Turn the repetition penalty off for it anyway: JSON requires repeating ", : and {, and a penalty over that punctuation is the one place it can actively break the format. The 0.6B-Chat scored 0/8 on the same prompts at every setting; route schema-shaped work here.
Why 512 and 1024 token budgets. These are measured, not taste. At a 150-token cap the chat slice terminated 0/2 in every cell and every long-form generation hit the cap. The 8B is verbose; give it room.
The repetition penalty is insurance, not a fix. Loop rate was 0.000 in all four cells, with and without it. It is in the conversation row to match what the behaviour receipts were graded at, and it is off everywhere reproducibility matters.
top_p 0.95 on the long-form row is the one unmeasured number here — we swept top-p 1.0 only. It is the conventional pairing for temperature 0.7, flagged as convention rather than measurement.
Top-k is not exposed on any runtime we ship, and this repo's generation_config.json sets none. Top-p is the only shortlist knob.
--stop-id 151645 is mandatory on the C binary. Without it the binary runs in fixed-length "race mode", ignores EOS and emits exactly the token count you asked for.
Speculative decoding and the exactness claim. Token-identical output is certified under greedy decoding (--temperature 0). With sampling, drafted output is drawn from the same distribution as undrafted output — the pip path routes through the transformers rejection-sampling correction — but individual tokens will differ. The research CUDA engine and the MLX --spec mode use argmax-match acceptance and are greedy-only. Device scope for the pip torch path (measured 2026-07-30, L4): the identity guarantee is CPU/greedy. On CUDA the activation dtype is bfloat16, and the batched verify pass can pick a different argmax winner than sequential plain decode at near-tie positions — one flip then a diverging tail (28 divergences / 232 tokens across the 8-prompt battery, 2 of 8 prompts affected). Treat pip CUDA drafting as same-distribution, not same-tokens, and run fermion verify --model <8B> --draft <0.6B> --device cuda on your own pair: it exits nonzero on FAIL, so it works as a gate in scripts. There is also no measured speed upside on that path (0.99x wall on an L4), so plain decode is the better default there.
Exactly one repetition penalty ever applies on the torch path: the fermion CLI pins repetition_penalty=1.0 in its generate() call before installing its own windowed processor, so HF's full-context processor is never built alongside it. If you assemble your own call, do the same — two penalties of 1.05 compose to a measured 1.1025.

`bin/` — prebuilt native runtimes (the fast path)

This repo bundles the prebuilt, closed-source fermion-run binaries next to the weights (the GGUF-ecosystem pattern; HF permits arbitrary binaries):

Binary	Platform	Backend
`bin/fermion-run-macos-arm64`	macOS arm64 (M1+)	CPU — NEON dotprod, zero dylib deps (Metal runtime is a week-one follow-up)
`bin/fermion-run-linux-x64`	Linux x86-64	CPU — AVX2 baseline, single-file binary (needs glibc ≥ 2.34)
`bin/fermion-run-linux-cuda`	Linux x86-64 + NVIDIA	CUDA — coming soon (not yet staged; today's CUDA fallback is the pip torch path)

Every binary ships with a <name>.sha256 sidecar (it holds a bare filename, so run the check from inside bin/); hf download writes files 0644, so mark the binary executable before running:

bash

(cd bin && shasum -a 256 -c fermion-run-macos-arm64.sha256)
chmod +x bin/fermion-run-macos-arm64
xattr -d com.apple.quarantine bin/fermion-run-macos-arm64 2>/dev/null || true  # macOS only

pip install fermion-research remains the one door: the CLI downloads the model and the platform-matching binary from THIS repo and shells out to it for the fast path (the torch reference path stays as the fallback). Until the auto-binary CLI lands, the documented one-command binary path is in bin/README.md.

Shipped-binary speed, with venue: 24.94 tok/s CPU-only on an M5 MacBook (the macOS arm64 binary, 9 threads, 32-token greedy, banked 2026-07-25) and 25.0 tok/s median via the MLX pack on a 16 GB M5 under a 6 GiB memory cap (mlx/receipts/). These are shipped-binary numbers; our research-harness results are reported separately and never conflated with them.

Native speculative decode (--draft), measured 2026-07-30. The Linux x86-64 binary pairs this 8B with a 0.6B draft container for in-process speculative decoding: 2.23× over plain (5.95 vs 2.67 tok/s, median of 3 runs, Linux x86-64 CPU container, 16 threads, 17–23% draft acceptance; measured with the 0.6B-Chat container as the draft). Pass a draft to get the multiplier; plain decode is unchanged. The recommended pairing is Neutrino-0.6B, whose weights are the distilled draft specialist as of 2026-07-31 — see the Speculative decoding section below. The same batched kernel also speeds plain batched prefill ~14× (automatic, no flags; 2026-07-30 campaign). macOS-arm64 native --draft is correctness-gated but pairing-limited today; no macOS speculative multiplier is quoted until it is re-measured. Receipts & fine print: receipts/rebuild_gate/lane_b.json — exactness-gated: the non-spec token stream is byte-identical to the shipped binary (ids certified; logits dump not exercised), and self-draft (the 8B drafting for itself) accepts 100% with an identical stream. Exact by construction: speculation moves the clock, never the text.

Binaries are closed-source and covered by the EULA line below; speed receipts (venue+version+date on every number) are retained in our internal evaluation records. bin/MANIFEST.json in this tree is the staging manifest: it names exactly which binaries land at upload time and their provenance.

Benchmarks

Everything here is non-thinking: all our numbers, and every same-harness cell we publish for another model, are measured with thinking disabled. Release battery: the shipped brain, graded on our house harness; all five release guards (termination, IFEval floor, Redux, knowledge-delta, C4 ppl) passed before the container was baked. A receipt exists for every cell (venue, version, date) in our internal evaluation records.

Release battery of the ship brain, protocol on every row:

Meter	Score	Protocol
MMLU-Redux	67.84	generative, thinking off
IFEval, prompt-strict	73.17	generative, thinking off
IFEval, instruction-strict	80.22	same run, per-instruction grading
IFEval, prompt-loose	76.31	same run, loose extraction
BFCL v3	65.31	macro over 13 subsets, bfcl-eval 2025.10.27.1, thinking off
GSM8K, flexible-extract	51.00	0-shot generative, greedy, 256-token cap
GSM8K, strict/stated format	49.33	same run, answer must appear in the stated format

Same-harness reference points for the strongest same-byte-class open model (Ternary-Bonsai-8B, measured by us on identical protocols): MMLU-Redux 71.47 · IFEval prompt-strict 83.65 · BFCL v3 71.45 · GSM8K flexible 35.00 · GSM8K stated-format 39.67. It leads us on instruction following, tool calling and MMLU-Redux; we lead on GSM8K flexible extraction (51.00 vs 35.00) and on GSM8K stated-format extraction (49.33 vs 39.67).

Reproducing the battery

The model registers as a native transformers model (import fermion), so the public harnesses run directly. GSM8K was graded with lm-eval:

python

# first: hf download fermionresearch/Neutrino-8B --local-dir Neutrino-8B  (see Quickstart)
import fermion, lm_eval
from lm_eval.models.huggingface import HFLM
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("Neutrino-8B")   # the downloaded directory
tok = AutoTokenizer.from_pretrained("Neutrino-8B")
lm = HFLM(pretrained=model, tokenizer=tok)
lm_eval.simple_evaluate(model=lm, tasks=["gsm8k"], num_fewshot=0)  # flexible-extract

IFEval, MMLU-Redux, and BFCL v3 were graded with evalscope 1.4.2 against a vLLM OpenAI-compatible endpoint serving this model, with chat_template_kwargs: {"enable_thinking": false} and bfcl-eval==2025.10.27.1 pinned:

bash

pip install evalscope==1.4.2 bfcl-eval==2025.10.27.1
evalscope eval --model Neutrino-8B --api-url http://127.0.0.1:8000/v1 \
    --datasets ifeval mmlu_redux bfcl_v3

The exact per-leg configs (context lengths 8k/16k, generation configs, the BFCL 13-subset list) and every raw report JSON are in the internal evaluation records.

Meter	Neutrino-8B	Same-harness comparison	Receipt
GSM8K flexible-extract (0-shot generative, greedy, 256-token cap, fixed 300-item subset)	51.00	Ternary-Bonsai-8B 35.0 on the byte-identical meter	internal evaluation records
GSM8K stated-format (same run, answer must land in the prescribed format; 1024-token cap)	49.33	Ternary-Bonsai-8B 39.67 on the byte-identical meter — this column is ours as of this brain	internal evaluation records

Don't-regress guards, held at release (guards, not headlines): GSM8K generative termination 0.51 — above the bf16 teacher's 0.42 anchor on the same raw-completion meter — and C4 perplexity 21.48 (base 21.40 — flat). The raw-completion protocol suppresses EOS across model families, ours and fp16 alike; chat-templated serving terminates normally.

Tool calling is a trained, measured capability of this brain, stated here as our own absolute numbers (no comparison rows on this axis): BFCL v3 13-subset macro 65.31 and IFEval prompt-strict 73.17 (bfcl-eval 2025.10.27.1, our harness, thinking off).

Weights: get them and verify them

The container ships in this repository. Fetch it and check the digest — MANIFEST.json carries the same sha256, and fermion info verifies it for you automatically:

bash

hf download fermionresearch/Neutrino-8B neutrino-8b_v4.bin --local-dir .
shasum -a 256 neutrino-8b_v4.bin         # must print 016c6f362...b4155fa0

Product bake provenance: built from Qwen3-8B via ternary QAT and staged post-training by Fermion Research, exported to the shipped TRTC v4 container.

Runtime matrix

Single-stream decode, one artifact across every row (venue + date on each number in the internal evaluation records):

Platform	Surface	Rate	Memory
H100 80 GB	research stack, drafted (0.6B draft, certified exact)	763 tok/s (counting; 402-500 tok/s on other prompt classes)	—
H100 80 GB	research stack, plain greedy	396 tok/s	—
NVIDIA L4	GGUF pack via our CUDA fork, full offload (rebuilt batched kernel)	35.4 tok/s plain decode	4.68 GiB @ 4k ctx
Apple M5, 16 GB	MLX pack (median, 3 runs)	25.0 tok/s	3.67 GiB peak, 6 GiB cap
Apple M5	`bin/` native binary, CPU only, 9 threads	24.94 tok/s	< 8 GiB resident

The L4 rate is plain decode on the rebuilt batched-verify CUDA kernel: tg128 35.40 ± 0.48 tok/s, +17% over the prior fork build's 30.27 ± 0.20 (NVIDIA L4, driver 580.95.05, llama-bench -r 3, fresh-clone same-container before/after, 2026-07-30). The bonus is automatic — no draft flag, no new options; every user of the rebuilt fork gets it. Speculative decoding on this pack is documented in gguf/README.md. Receipts & fine print: receipts/rebuild_gate/lane_a.json — the rebuild is exactness-gated (test-backend-ops FV5/FV5B 0 FAIL on CUDA and CPU; 128-token greedy completion token-identical before → after and CUDA vs CPU).

Speculative decoding — pair with Neutrino-0.6B (distilled draft)

The draft for this 8B is Neutrino-0.6B. Since 2026-07-31 that repo's weights are a draft specialist: a 0.6B five-value container distilled from this 8B's rollouts so its greedy predictions match the 8B as often as possible. It exists to make the 8B faster, not to chat: pair it with this model via fermion chat --draft, the MLX pack's --spec, or llama-speculative -md on our llama.cpp fork. Draft and target are the same TRTC v4 container format, read by the same runtimes, in one process — no extra conversion, and every existing draft pointer (the 0.6B container and gguf paths) keeps working. The 0.6B's prior cut remains available via revision pinning on that repo.

With greedy decoding, drafted output is token-identical to plain Neutrino-8B decode — speculation moves the clock, never the text (certificate in the receipts line below).

Measured with the distilled draft in the harness (H100, 3-round median, dynamic draft length, 2026-07-30):

prompt class	agreement with the 8B (teacher-forced, held-out)	end-to-end speedup
counting	100%	×1.80
facts	~87%	×1.23
prose	~75%	×1.12
chat / explanation	~75%	×1.07
code	~78%	×1.06
forced-reject floor	—	×0.75 (output exact)

That agreement is up from 44–82% for the base-0.6B function this draft replaces as the recommended pairing (same held-out probe, 2026-07-30). One disclosed sub-bar cell: a technical-explanation slice measured 61.9% agreement against its 65% bar (end-to-end ×0.91 there). Apple-silicon (MLX --spec) door numbers ship in receipts/ with venue + version pins; that path is verify-kernel-bound today and improves with the in-flight Metal kernel work. The llama.cpp-fork and native-binary speculative numbers (CPU, CUDA, Linux) live in gguf/README.md and the bin/ section above.

The draft is not a chat model. It answers as a compressed echo of this 8B's next-token habits; for conversation use Neutrino-0.6B-Chat or this 8B. Speculative decoding is a throughput feature with a correctness certificate; it does not change quality, and the speedup is prompt-dependent — the rows above are the honest spread, not a single headline number.

Receipts & fine print (exactness-gated): receipts/draft/cert_20260730.log — 27,648 spec tokens replayed against exactly this pairing (this 8B container + the 2026-07-31 Neutrino-0.6B weights), 0 mismatches (H100, 2026-07-30; greedy draft+verify vs plain greedy); receipts/draft/acceptance_20260730.log — the per-class agreement table; receipts/dspec_cert_shipbrain.json — the earlier ship-brain cert (2026-07-27, 27,648 tokens, 0 mismatches, prior 0.6B draft cut) retained for continuity. Under sampling, drafted output is same-distribution rather than same-tokens — see the device-scope note in Recommended settings.

Notes

The GGUF pack loads only through our llama.cpp fork (gguf/) until upstreaming lands; stock llama.cpp, ollama, and LM Studio binaries do not know the FV5/FV5B tensor types.
This is a non-thinking model: it ships and is evaluated with thinking disabled (enable_thinking=False).

Model lineage and methodology

Built from Qwen3-8B via ternary QAT and staged post-training by Fermion Research; the engineering story of the format and the engine is on the research pages at fermionresearch.com/research/.

License and attribution

Weights: Apache-2.0. Derivative of Qwen/Qwen3-8B (Apache-2.0, Alibaba Cloud) — see LICENSE. Post-training data sources are disclosed in the internal evaluation records (UltraChat, MetaMath, and the other accepted-source datasets).

bin/ binaries: prebuilt, closed-source, free to use with these weights; no redistribution outside this repo; no reverse engineering.

Contact: contact@fermionresearch.com

FermionResearch/Neutrino-8B

Neutrino-8B

Architecture

Formats and artifacts

Quickstart

Recommended settings, by use case

bin/ — prebuilt native runtimes (the fast path)

Benchmarks

Reproducing the battery

Weights: get them and verify them

Runtime matrix

Speculative decoding — pair with Neutrino-0.6B (distilled draft)

Notes

Model lineage and methodology

License and attribution

FermionResearch/
Neutrino-8B

`bin/` — prebuilt native runtimes (the fast path)