Unary Quantization Research

Overview

Contents

Converters (Python)

• unary_convert.py / unary_convert_v2.py — Base unary thermometer conversion
• convert_proper_unary.py / convert_proper_unary_v2.py — Proper unary with group quantization
• convert_log_unary.py — Log-spaced unary variant
• convert_fast.py — Optimised conversion pipeline
• packed_convert.py / packed_loader.py — Packed binary format
• convert_qwen3.py / convert_qwen3_v2.py — Qwen3-4B specific converters

C Inference Engines (AVX-512 + POPCNT)

• unary_engine.c / unary_engine_v2.c — Core unary inference
• pure_unary_engine.c — Pure unary (no FP in linear layers)
• log_unary_engine.c — Log-unary engine
• proper_unary.c — Proper unary with group scales
• true_unary.c — True base-1 unary engine
• concat_unary.c — Concatenated unary engine
• packed_engine.c — Packed bitplane engine
• unary_full.c — Full forward pass engine

Converted Models

• deepseek-r1-1.5b-* — DeepSeek-R1-1.5B in multiple unary variants (4-plane, 7-plane, 31-plane, grouped, packed, ternary baseline)
• qwen3-4b-* — Qwen3-4B-Thinking in unary, log-unary, and proper-unary variants

Benchmarks and Runners

• bench_fwd.py / bench_gen.py / bench_prompt.py — Performance benchmarks
• inference.py / server.py — Python inference and API server
• Various run_*.py — Model-specific runners

Key Insight

Building

License