Qwen3.5-0.8B — fni8 (int8/W4A8 dp4a, Volta sm_70)

⚠️ EXPERIMENTAL — do not rely on this checkpoint.

It was converted before the hybrid-config-drop converter fix: the hybrid (Gated-DeltaNet) portion of the config was dropped during conversion, so the file may not load or run correctly. A corrected reconversion is pending. Even once reconverted, full decode additionally requires the Track-2 DeltaNet int8 kernel (currently a fp16 linear-attention fallback).

Status

Format

• Weights: int8 per-row (W8A8) or int4 per-group (W4A8), fp32 scales, resident dp4a VRAM layout.
• Why dp4a: sm_70 has no int8 tensor cores, so the matmul contraction runs on the __dp4a CUDA-core intrinsic. On the CMP 100-210 fleet (whose fp16 tensor cores are firmware-limited) dp4a is the fast path, not a compromise.

How to run

Limitations

• Quantization is lossy: int8 (and especially int4) outputs differ from the fp16/bf16 parent, and the difference varies by task.
• Capabilities, biases, and risks of the parent model carry over — see the parent card.
• This is a derivative quantization, not a relicense; the parent model's license and acceptable uses apply.