A 1.58-bit BitNet-style sentence embedding model distilled from sentence-transformers/all-MiniLM-L6-v2 via quantization-aware training, with post-training int4 quantization at the embedding layer. Weights are ternary {-1, 0, +1}, so inference is adds and subtracts rather than float matmuls, and the whole model — engine, tokenizer, and weights — ships as a single WASM bundle that runs on CPU with no API calls, no GPU, and no runtime download.

ternlight ships in two tiers, same API and same 384-dim output — pick by the size/quality trade. It is designed for short-string semantic similarity (search queries, intent classification, FAQ matching, product cards) deployed on-device (browser, Node, edge runtimes, ARM single-board computers). It is not a frontier model; it trades absolute quality for size and on-device deployability.

Tiers

Tier	Architecture	Params	Wire (gzip)	Latency (p50, CPU)	Throughput	Spearman vs teacher	SciFact NDCG@10
base ⭐	2-layer · d_model 384 · 6 heads	~15.4M	7.2 MB	5.1 ms	~195 emb/s	0.844	0.465
mini	2-layer · d_model 256 · 4 heads	~9.5M	5.0 MB	2.5 ms	~400 emb/s	0.820	0.439
teacher (MiniLM-L6)	6-layer · d_model 384	~22.7M	~90 MB (fp32)	—	—	1.000 (ref)	—

base — the quality tier. ~12× smaller on the wire than the fp32 teacher while retaining 0.84 rank-correlation. Use when quality matters more than the last 2 MB.
mini — the small/fast tier. ~1.6× the throughput at 5 MB, a modest quality step down.

Both tiers: ternary linear weights + int4 embedding table, 384-dim L2-normalized output, 128-token max input, BERT WordPiece vocabulary (30,522, identical to the teacher). Numbers measured on the shipped int4 builds (M-series Mac, Node single-threaded).

How to use

ternlight runs via a custom Rust→WASM inference engine, not via the transformers library. Two paths:

Path 1 — via npm (recommended)

bash

npm install @ternlight/base    # quality tier — 7 MB wire, ~5 ms/embed
npm install @ternlight/mini    # small tier   — 5 MB wire, ~2.5 ms/embed

import { embed, cosineSim, similar } from '@ternlight/base';   // or '@ternlight/mini'

const v1 = embed("arctic terns migrate from pole to pole");
const v2 = embed("longest migration in the animal kingdom");
cosineSim(v1, v2);   // ~0.71 — semantically related, different wording

// Nearest-neighbor search over a corpus
similar("which seabird travels farthest", corpus, { topK: 5 });

The model and tokenizer are bundled into each npm package — no separate download. Works in Node ≥ 18, browsers (via any bundler), Cloudflare Workers, Vercel Edge, Deno, and Bun.

Path 2 — direct download

python

from huggingface_hub import hf_hub_download

# pick a tier (adjust the paths to match this repo's file layout)
model_bin = hf_hub_download(repo_id="wenshutang/ternlight", filename="base/model-int4.bin")
tokenizer = hf_hub_download(repo_id="wenshutang/ternlight", filename="tokenizer.json")

The .bin files are a custom BitNet b1.58 format. See the engine source for the binary layout and reference forward pass if you want to implement a custom loader in another language or runtime — there is no transformers.AutoModel.from_pretrained() path.

Model details

Property	base	mini
Layers	2-layer Transformer encoder	2-layer Transformer encoder
d_model	384	256
Attention heads	6	4
Parameters	~15.4M	~9.5M
Output dimension	384 (L2-normalized)	384 (L2-normalized)
Max input	128 tokens (~95 English words; longer inputs are silently truncated)	same
Vocabulary	30,522 (BERT WordPiece, identical to teacher)	same
Linear weights	Ternary `{-1, 0, +1}` + per-matrix fp32 scale	same
Embedding table	4-bit per-row PTQ + per-row fp32 scale	same

Training

Distilled from sentence-transformers/all-MiniLM-L6-v2:

Distillation objective — cosine/MSE loss between student and teacher 384-dim embeddings over ~1M sentence pairs (search queries, paraphrases, statements).
BitNet b1.58 quantization-aware training — all linear layers use ternary weights trained end-to-end with the straight-through estimator. Training under the quantization constraint from the start (rather than quantizing post-hoc) is what preserves fidelity: a naive post-training ternary quant of the same encoder drops sharply, so nearly all of the retained quality comes from the QAT, not the format.
Post-training int4 quantization — applied to the token embedding table after QAT, chosen via an ablation over int8/int4/ternary on the table. The embedding table dominates parameter count, so compressing it gives the largest size win for the smallest quality cost.

Training data (base): mix_v3_1M — ~1M pairs from MS MARCO, Quora duplicates, AllNLI, GooAQ, and StackExchange duplicates (English-only, seed=42). See configs/mix-v3-robust.yaml.

Provenance

base (model-int4.bin)


Training run	`robust-d384-mixv3-ep40`
Source checkpoint	`checkpoint_ep40.pt`
Source code commit	`178e227`
Data manifest	`mix_v3_1M` (seed 42)
Packed at	2026-07-05
Bin size	7,492,312 bytes
SHA-256	`68cc2c43…d2ee1db8`
Gate (2026-07-04)	test/spearman 0.851, ndcg@10 0.4665 (int8 form)

mini (model-int4.bin) — ⚠️ verify against the currently shipped @ternlight/mini build; this sidecar predates the 2026-07-05 package rebuild.


Training run	`qat-resume-ep10-ep40`
Source checkpoint	`checkpoint_ep40.pt`
Source code commit	`dff16b1`
Packed at	2026-06-03
Bin size	4,839,512 bytes
SHA-256	`07d8cfdb…f2e5b6c98`

Each .bin ships with a .bin.json sidecar containing full provenance for reproducibility.

Evaluation

Fidelity — Spearman rank correlation vs teacher. Held-out queries, 1,000 deterministic random pairs, seed=42. Spearman of 1.0 = ranks pair similarities identically to the teacher.

Retrieval — SciFact NDCG@10. Absolute retrieval quality on the BEIR SciFact task.

Model	Spearman vs teacher	SciFact NDCG@10
MiniLM-L6 (teacher)	1.000 (ref)	—
ternlight base ⭐	0.844	0.465
ternlight mini	0.820	0.439

Full methodology and reproduction scripts: eval/quality/RESULTS.md.

Intended use

Designed for:

Short-string semantic similarity (queries, intents, FAQs, product titles, tags)
On-device deployment — browsers, Node services, Cloudflare Workers, Deno Deploy, Vercel Edge, Raspberry Pi-class ARM single-board computers
Cost-free embedding at any scale (no per-call API charges)
Privacy-sensitive workloads where queries cannot leave the user's device

Not designed for:

Long-document understanding (max input is 128 tokens — silently truncated above)
Multilingual workloads (English-only, inherited from MiniLM-L6)
Maximum absolute quality (use a frontier model like text-embedding-3-large or voyage-3 if quality dominates over size and deployability)

Limitations

English-only: teacher, tokenizer (bert-base-uncased, no CJK vocab), and training data are English. Non-English text will not tokenize or embed sensibly. Multilingual is the most-requested feature and the pipeline is language-agnostic, but it is not done yet.
128-token cap: text longer than 128 BERT WordPiece tokens is silently truncated. Embed at sentence or short-paragraph granularity, not full document.
Custom runtime required: no transformers path. Use the npm packages or implement a custom loader from the binary format.
Inherited biases: distilled from all-MiniLM-L6-v2; the same demographic and topical bias caveats from the sentence-transformers corpus apply.
v0.1: the binary format and JS API may change before v1.0.

License

MIT, matching the teacher model and the ternlight project. See LICENSE.

Citation

bibtex

@software{ternlight2026,
  title  = {ternlight: a 1.58-bit BitNet sentence embedder in a few MB of WASM},
  author = {Tang, Wen Shu},
  year   = {2026},
  url    = {https://github.com/soycaporal/ternlight}
}

ternlight builds on:

BitNet b1.58 (Ma et al., 2024) — ternary weight training
bitlinear by @schneiderkamplab — the reference PyTorch implementation of BitLinear, used directly during training (bitlinear==2.4.6); the Rust inference engine mirrors its forward-pass math byte-for-byte
sentence-transformers/all-MiniLM-L6-v2 — teacher model

wenshutang/ternlight

ternlight