Views
No views yet
mimo_v2.py — PrismaQuant MiMoV2Profile (nested per-expert MoE, fused qkv, FP8-source, 3 dense MTP layers). Verified: detect_profile→MiMoV2Profile, 256 experts collapse per layer, dense-0 scored separately.mimo_v2.json — declarative model-structure spec.registry.patched.py — profile registration (before DeepseekV4 + DefaultProfile).calib/*.jsonl — domain calibration set (JSON/tool-call/tender-extraction), disjoint gate split.step1_keymatch.py — state-dict key-match diagnostic.configuration_mimo_v2.py, no modeling_*.py; auto_map=AutoConfig only).
transformers v5.0 built-in MiMoV2FlashForCausalLM (model_type mimo_v2_flash) is STRUCTURALLY INCOMPATIBLE:
batched-3D experts (vs 72,192 separate per-expert tensors), split q/k/v (vs fused qkv_proj), no MTP.
=> A vendored modeling_mimo_v2.py (name-matching the checkpoint: nested 256-expert ModuleList w/ separate
gate/up/down_proj; fused qkv_proj + attention_sink_bias on 39 SWA layers; model.mtp.layers.{0,1,2}; dense layer-0)
is required for the transformers-based tooling path. MXFP4 being data-free also permits a direct tensor-level
requant that bypasses model-loading. dflash/dflash.py may seed the MTP/attention shim.