Runtime: available. A complete, self-contained vLLM runtime for 2× DGX Spark (GB10, sm_121) is published and validated — see Runtime. GitHub: https://github.com/jjang-ai/dots3-note-gb10-vllm · Image: ghcr.io/jjang-ai/dots3-note-gb10-vllm:sm121-cu130.

What this is

An NVFP4 requantization of dots-studio/dots3-note-prev, targeted at NVIDIA Blackwell (RTX 50, GB10 / DGX Spark) via vLLM's native modelopt_fp4 path.

Routed experts → NVFP4 (E2M1, group-16, FP8-E4M3 block scales + FP32 global scale — modelopt format).
Kept high-precision (BF16): MLA/DSA attention, shared expert, router gates, the dense layer 0, the MTP / NEXTN draft head (layer 46), and the full vision (MoE-ViT) + audio encoders. Multimodal and speculative decoding are preserved, not stripped.
hf_quant_config.json: quant_algo=NVFP4, 10 exclude patterns (attention / shared / dense-0 / MTP / vision / audio / lm_head).
~174 GB on disk. Fits TP2 across two 121 GB GB10 nodes.

Property	Value
Base	dots-studio/dots3-note-prev (Apache-2.0)
Total / active params	280B / 16B
Experts	256 routed + 1 shared, top-8
Attention	13 DSA (top-2048) + 33 SWA
MTP / DSpark	1 shared NEXTN head, 1.13B (layer 46) — kept BF16
Vision / Audio	MoE-ViT 7B (1.2B act) / dense 800M — kept BF16
Context	up to 512K (DSA-native `fp8_ds_mla` paged KV)
Expert precision	NVFP4 (E2M1, g16)
Input / Output	text · image · video · audio / text

Recommended generation config

Defaults tuned for agentic / tool-use / coding work (not plain instruct chat).

python

from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:8001/v1", api_key="EMPTY")

resp = client.chat.completions.create(
    model="dots3-note",
    messages=[{"role": "user", "content": "Refactor this module and add tests."}],
    temperature=1.0,          # upstream-recommended sampling
    top_p=0.95,
    max_tokens=8192,          # give agentic/coding turns room
    extra_body={
        # Reasoning control (dots3 is a boolean thinking toggle, not tiered effort):
        #   enable_thinking=True  → deliberate multi-step reasoning (hard reasoning / agent planning)
        #   enable_thinking=False → direct answer (low latency, simple tools)
        "chat_template_kwargs": {"enable_thinking": True},
    },
)

Setting	Recommended	Notes
`temperature`	1.0	upstream default; lower (0.2–0.6) for deterministic code edits
`top_p`	0.95
`enable_thinking`	True for agents/hard reasoning, False for simple/latency-sensitive	dots3 has no separate low/med/high tiers — this boolean is the reasoning switch
`max_tokens`	≥4096 (8192 for coding)	thinking + tool loops need headroom
repetition penalty	none	not recommended for this model

Agentic tool calling is native. When serving, enable --enable-auto-tool-choice --tool-call-parser dots; the chat template emits/parses <tool_call> blocks. Pass your tools=[...] schema as usual — no custom system prompt needed.

Multimodal (image · video · audio)

python

messages = [{"role": "user", "content": [
    {"type": "video_url", "video_url": {"url": "https://.../clip.mp4"}},
    {"type": "text", "text": "Describe the performance and what can be heard."},
]}]

Video inputs carry their audio track — the model reasons over both jointly.
Image / audio / video all route through the preserved (BF16) encoders; multimodal quality is unaffected by NVFP4 expert quantization.
The preprocessor_config.json for vision/video/audio is included.

Runtime — available

A complete, self-contained vLLM runtime for 2× DGX Spark (GB10, sm_121, CUDA 13, aarch64) over TP2:

GitHub: https://github.com/jjang-ai/dots3-note-gb10-vllm
Docker: ghcr.io/jjang-ai/dots3-note-gb10-vllm:sm121-cu130

DeepSeek Sparse Attention (DSA) is not officially supported on consumer / GB10 Blackwell (sm_120/sm_121) in upstream vLLM — the fused sparse-attention and top-k CUDA kernels are Hopper / datacenter-Blackwell only and crash or emit garbage on GB10. This runtime carries the patches that make it correct and fast:

DSA sparse-MLA decode + indexer → exact torch references (the deep_gemm / FlashInfer sm120 kernels are numerically wrong on GB10 across a KV-block boundary).
DSA top-k → torch.topk on sm_121 (the fused kernel needs ≥128 KB shared memory; GB10 exposes ~99 KB, so it crashes for any large max_model_len). Mirrors upstream vLLM PR #49897.
NVFP4 via Marlin W4A16 (the FlashInfer / CUTLASS FP4 GEMMs no-op on sm_121).
CUDA graphs (FULL_AND_PIECEWISE) for ~1.8–2× decode; NCCL 2.30.4 preload to fix a dual-Spark deadlock; --skip-mm-profiling so multimodal and a large KV pool coexist.

Full root-cause writeup: docs/GB10_FIXES.md.

Validated performance (2× GB10, driver 580.x, TP2)

Metric	Value
Decode (MTP + CUDA graphs)	~24–25 tok/s (vs ~14 eager)
Prefill	>1000 tok/s past ~330k ctx (prefix reuse); decode ~flat in context length
Max coherent context	428,671 tokens — single conversation, 0 errors (KV-pool bound)
Multimodal	image · video · audio all verified
Reasoning	default on; `enable_thinking` toggle per request

Serve (Docker, both nodes)

bash

docker pull ghcr.io/jjang-ai/dots3-note-gb10-vllm:sm121-cu130
# worker on spark2 FIRST, then head on spark1 (host networking; set your node IPs + RoCE iface)
IMAGE=ghcr.io/jjang-ai/dots3-note-gb10-vllm:sm121-cu130 MODEL_DIR=/path/to/dots3-note-prev-NVFP4 \
  HEAD_IP=10.99.0.1 WORKER_IP=10.99.0.2 bash docker/run_worker.sh   # spark2
IMAGE=ghcr.io/jjang-ai/dots3-note-gb10-vllm:sm121-cu130 MODEL_DIR=/path/to/dots3-note-prev-NVFP4 \
  HEAD_IP=10.99.0.1 WORKER_IP=10.99.0.2 bash docker/run_head.sh      # spark1

The container defaults to the fast config (MTP + CUDA graphs + NCCL preload). Every env knob and the from-source patch (patches/apply.sh, no image needed) are documented in the GitHub repo.

Access

Open weights — no gating. Apache-2.0, inherited from upstream dots-studio/dots3-note-prev.

Credits

Base model: dots-studio — dots3-note-prev.
NVFP4 requantization + GB10/TP2 runtime: JANGQ-AI.

JANGQ-AI/dots3-note-prev-NVFP4

dots3-note Preview — NVFP4 (JANGQ-AI)