FP8 W8A8 build of baidu/Unlimited-OCR (revision 07dea832), the 3.34B DeepSeek-V2-style MoE OCR model ("one-shot long-horizon parsing"), validated against a BF16 baseline of the published checkpoint. All credit for the model belongs to Baidu; this repo changes only the numeric precision of the text-decoder weights. Checkpoint size drops from 6.7 GB to 3.9 GB.

Pick this variant for Ada/Hopper/Blackwell GPUs where you want near-lossless behavior with native FP8 execution. The companion NVFP4 build — calibrated, unlike prior community 4-bit builds — shrinks the footprint further for memory-constrained Blackwell deployments.

Scheme: FP8_DYNAMIC via llm-compressor — per-channel FP8 weights, dynamic per-token FP8 activations, data-free. Only the DeepSeek-V2 MoE text-decoder linears are quantized (2196 modules: 2112 routed-expert, 33 shared-expert/dense-MLP, 48 attention, 3 dense-layer). Everything the vision path depends on stays BF16:

Greedy OCR on the fixture set (vLLM 0.26.0, SM120) is character-identical to the BF16 baseline — CER 0.0 on every fixture, grounding-box coordinates included. The same parity holds under transformers + compressed-tensors.

Raw per-fixture transcripts ship under evidence/, produced by the validation harness.

Note: the shipped quantization_config.ignore uses prefix-agnostic regex patterns — required for vLLM to correctly skip the vision tower for quantization compatibility. Do not replace it with expanded module names.

Quantized by Matt Busi at ShadowRock. Raw evaluation outputs ship under evidence/.