This repository contains a highly optimized TQ1 quantized version of the official microsoft/bitnet-b1.58-2B-4T model for the ATLAS Engine ecosystem, designed for native, ultra-low-latency CPU inference without any GPU requirement.

Packed using the unified pack_to_atlas.py toolchain (v2.10.0) with BF16 weight scale correction.

Engine Specifications

Property	Value
Format	ATLAS Binary (`.atlas`), format_version=2
Quantization	TQ1.0 — Ternary Weight Packing (Base-3, ~1.58 bits/weight)
Target	Native CPU — Intel AVX2 (Haswell 2013+), no GPU needed
File Size	1.04 GB
Inference Speed	~2.8 tok/s (i7-7700T, f32 bypass)
Description	BitNet b1.58 2B-4T, SubLN architecture, Microsoft pre-quantized U8 weights

Architecture

Component	Detail
Base Model	`microsoft/bitnet-b1.58-2B-4T`
Architecture	bitnet
Layers	30
Hidden Size	2560
Intermediate Size	6912
Attention Heads	20 (GQA, 5 KV heads)
Head Dim	128
RoPE Theta	500000.0
Vocabulary	128256
Context Window	2048 (NTK-scalable up to 4096)

Architecture Note: BitNet uses SubLN (sub-layer normalization) with ReLU² activation. This architecture requires f32_bypass mode (auto-enabled) because the ternary weights are ~0.01× scale, and uint8+128 activation quantization destroys the signal. The T=0 (argmax) path degrades into token repetition after 10-15 tokens due to ternary quantization — T=0.7 with top_k=40 is the stable default.

Verification

During pre-release evaluation (v2.10.0), this quantized derivative demonstrated correct convergence:

T=0 (argmax): "The capital of France is Paris." — correct deterministic output
T=0.7 (sampling): Coherent structured generation with sensible continuation

Note on scale mathematics: the legacy dequantization path divides by the scale factor rather than multiplying. Since this is a constant across all logits for any given output row, the relative probability distribution remains identical under softmax normalization — no effect on output quality.

Prompt Template

To prevent token degradation and alignment shifting, use the standard chat template:

{role}: {content}<|eot_id|>

Example Sequence

User: What is 2+2?
Assistant: 4

Usage

Python

git clone https://github.com/xxxn3m3s1sxxx/ATLAS-TQ1_0.git

python

from atlas_infer import AtlasModel

model = AtlasModel("BitNet-2B4T-b1.58-ATLAS.tq1.atlas")
output = model.generate_c(
    "What is the capital of France?",
    max_new_tokens=100,
    temperature=0.7,
    top_k=40,
)
print(output)

C++ CLI (standalone, no Python required)

atlas --model BitNet-2B4T-b1.58-ATLAS.tq1.atlas --prompt "What is the capital of France?" --max-tokens 100

SSE Web Server

bash

python atlas_server.py --model BitNet-2B4T-b1.58-ATLAS.tq1.atlas --port 8080
curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"prompt": "What is the capital of France?", "max_tokens": 100}'

What is ATLAS?

ATLAS is a CPU inference engine for BitNet b1.58 ternary-quantized models. It repacks HuggingFace safetensors into the TQ1.0 format (5 ternary trits per byte, Base-3 encoding, ~1.58 bits/weight) and runs fast inference via a C++ DLL + Python wrapper.

Feature	Description
No GPU required	Runs on any x86-64 CPU with AVX2 (Intel Haswell 2013+, AMD Excavator 2015+)
Hybrid matmul	FFN tensors in int8, QKV/O in TQ1-packed, per-tensor dispatch
int4 FFN mode	Halves FFN memory bandwidth for 18-26% speedup (7B/10B)
f32 bypass	Auto-enabled for small models (≤1B) and SubLN architectures
Ring buffer KV cache	Extended context via NTK-aware RoPE scaling
Standalone C++ CLI	No Python or PyTorch required at runtime
SSE web server	FastAPI-based `/v1/chat/completions` with prompt caching

License

This is a quantized derivative work based on the BitNet b1.58 architecture (original model by Microsoft), originally released under MIT License.

The ATLAS engine itself is Apache 2.0 licensed.

xxxn3m3s1sxxx/BitNet-2B4T-b1.58-ATLAS

BitNet-2B4T-b1.58-ATLAS (v2.10.0)