Almost every public NVFP4 checkpoint is W4A4. This one is weight-only: 4-bit weights, BF16 activations, no calibration corpus anywhere in its construction — and therefore nothing you would have to reproduce to rebuild it.

20.6 GB. Runs on one GB10 with images and video intact, and keeps the MTP head so speculative decoding works without a separate drafter.

You have an NVIDIA DGX Spark — GB10, SM121, one coherent ~121 GiB memory pool — and you want a 27B multimodal model that leaves room for real context. You feed it screenshots, photos, or video, not just text.

If you are on a discrete GPU, nothing here is wrong, but none of it was tuned for you. The choices below assume unified memory, where "GPU memory" and "host RAM" are the same DRAM and offloading reclaims nothing.

W4A16 and W4A4 store the same 4-bit weights. The difference is what else the build depends on. W4A4 quantizes activations too, and activation scales have to come from a calibration corpus — the checkpoint inherits whatever that corpus contained, and anyone rebuilding it has to reproduce the corpus to reproduce the model. Weight-only derives every scale from the weights themselves: rebuilding this takes the source checkpoint, quantize_w4a16.py from this repo, and nothing else.

Throughput does not break the tie. Decode on GB10 is bandwidth-bound: both schemes move the same 4-bit bytes, and this build dequantizes FP4 to BF16 rather than touching the FP4 tensor cores at all. Measured here: W4A4 decoded 20.9 tok/s against this build's 19.8, then gave the margin back on video prefill. Neither scheme has a throughput advantage worth choosing on.

Does W4A4 degrade image and video, as folklore says? No. We built one and tested it. Same source, ModelOpt stock configs on both sides, text-calibrated, same engine, same 24 prompts, same BF16 reference. Zero empty replies, every ground-truth string matched, and the burned-in subtitle transcriptions came back byte-identical to this build. 13 of the 24 replies differed, so the activation path was live rather than silently falling back to the weight-only kernel. That is one eval on one checkpoint with one calibration set -- it does not make W4A4 safe everywhere, but the failure it is famous for did not happen.

So weight-only is the simpler construction, not a rescue from a broken alternative. The compression is the win; the 4-bit math never was.

The launcher ships in this repo — there is nothing else to clone.

That gives you an OpenAI-compatible endpoint on :30000, with MTP speculative decoding, 262K context, and --mem-fraction-static 0.75.

If you would rather not run someone else's shell script, the whole thing is one docker run — this is what the script does, minus the readiness polling:

One more thing: do not copy --mem-fraction-static 0.95 from elsewhere. On one coherent pool that starves the operating system as well as the server, and a long video prefill will take the whole box down with a global OOM — measured here, exit 137, dbus killed too.

These are measurements from the machine that built this checkpoint, not projections.

Speculative decoding is tuned. MTP depth was swept 2/3/4/5 and 3/1/4 wins on code (23.4 / 25.5 / 23.6 / 21.2 tok/s), which is also the setting serve_export.sh ships. The mechanism is visible in the data: accept length climbs monotonically with depth (2.45 → 2.81 → 2.95 → 3.08) while throughput peaks at 3, because the MTP head is trained for exactly three steps and deeper chains only add verify work that gets rejected. If your workload is prose rather than code, 2/1/3 is slightly better (18.4 vs 17.3).

You will see higher figures quoted for GB10. Most are extrapolated from other hardware rather than measured on one — worth checking which you are reading.

MLP is about 62% of the parameters, which is why quantizing it and leaving everything sensitive alone gets most of the compression for very little of the risk. Produced with NVIDIA Model Optimizer's stock W4A16_NVFP4_CFG.

One packaging note if you are reading the files: hf_quant_config.json declares MIXED_PRECISION with a per-module map rather than a bare W4A16_NVFP4. That is deliberate. SGLang rejects the latter outright even though it has the matching kernel; the map is the only form that reaches it. ModelOpt's original config is preserved alongside as hf_quant_config.modelopt.json.

Smoke test: 6/6 cases passed (5 vision). Export verification: 1999 tensors, no activation scales, MTP head and vision tower present and unquantized.

Those are structural checks. They cannot tell you whether 4-bit weights cost real capability, because the realistic failure of a good quantization is output that is worse but still coherent, which nothing automatic catches. So:

24 prompts across text, images and video were run against both this checkpoint and BF16 source / SGLang, on the same engine so the weights are the only variable. No fatal flags were raised. Every ground-truth string the reference matched, this build matched too — including all 6 subtitle strings burned into 3 real video clips, read back intact.

If image or video quality matters to you, run that comparison on your inputs. eval_harness.py, make_evalset.py and serve_export.sh are all in this repo.

Quantization damage tends to surface in agentic work before it surfaces in benchmarks. A model that still writes fluent prose can stop honouring a tool schema, lose the thread over a long run, or quietly stop noticing things. Three runs, on the GB10, served exactly as serve_export.sh ships it.

Codebase recon against known ground truth. Trace how a capability-enforcement setting travels through an unfamiliar Rust workspace — where it is defined, where it is validated, where it actually takes effect — and cite file:line for every link. It got all three links exactly right, then went past what was asked: the filter implementation, the pre-spawn validation entry, the test locations, the CI commands. What it did with its own uncertainty mattered more. Its "unknowns" section flagged two things it could not confirm, and both were real defects: a struct field whose doc comment claims it is applied at spawn but which has no production consumer anywhere, and a validator reachable only from tests. Both confirmed by grep afterwards. 16m13s.

A feature, unattended, end to end. Explore → plan → implement → verify with no human in the loop: add a --dry-run flag to two installers, one sh and one PowerShell, plus the documentation for it. 70m38s. Re-tested independently afterwards on both platforms — dry run writes nothing and exits 0, normal install unchanged, uninstall still clean, unrecognised arguments still fall through to a normal install. Its own review claimed eight checks passed; all eight held up when they were re-run by hand. It wrote scratch test scripts as it worked and deleted them when it was finished.

Vision inside the loop. Handed a screenshot through the same tool path, it transcribed every line exactly, including the hex string 0x7F4A that a 12B vision model misread on two separate attempts. It also described the image's white background as black, and labelled that observation certain. Transcription is trustworthy here. Scene and colour description is not — on this model or on the 12B.

Same tasks, same box, one run each:

Laguna is about twice as fast generating code and finished the feature 25 minutes sooner. It did not notice either defect on the same recon task. Decode here is measured with one harness across all three so the column compares — it counts prefill in wall clock, which is why this build reads 20.9 rather than the 25.7 measured above.

Two things that matter to a harness and are easy to get wrong: with --tool-call-parser qwen3_coder this build returns structured tool_calls, and with --reasoning-parser qwen3 the reasoning lands in reasoning_content with nothing leaking into content. Both are already set by serve_export.sh.

Read this for what it is: one run per model per task, one harness (Axon driving oh-my-axon), against a codebase whose ground truth was known in advance — which is what makes the recon result checkable, and also what keeps it from being a standard benchmark. Unlike everything else on this page, these numbers cannot be re-derived from this repo alone.

Qwen/Qwen3.8-27B → DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1 → this.

Finetune by DavidAU (COLD FUSION / GAIN + Unsloth). Base model by Qwen.

The full quantization and evaluation pipeline is in this repo: quantize_w4a16.py, verify_export.py, smoke_test.py, eval_harness.py, serve_export.sh, spec_sweep.sh. Everything asserted on this page can be re-derived with them.

Apache 2.0, unbroken from the base model.