DeepSeek-V4-Flash-0731 GGUF — Quantized by BatiAI

The 2026-07-31 refresh of DeepSeek-V4-Flash — quantized from the official weights, verified in Korean. 284B-A13B MoE with CSA+HCA hybrid attention. Runs on a single high-RAM Mac.

📦 Quantizations

🆕 What changed in 0731. Same architecture as the original V4‑Flash (43 layers, 256 experts, vocab 129280) with refreshed weights — and a new storage format: experts are stored INT8 with UE8M0 scales (scale_fmt: ue8m0) instead of plain FP8. That is why these files are slightly larger than our earlier V4‑Flash GGUFs, and why conversion needs torch>=2.7 (E8M0 dtype).

✅ Verified (this build, Q3_K_M)

🚀 Usage

⚙️ Runtime: batiai/bati.cpp — DeepSeek‑V4's CSA+HCA hybrid attention (deepseek4) is not in mainline llama.cpp. Build our fork.

⚠️ Two things the base repo doesn't tell you (we hit both)

1. No chat template ships with this model — neither the original nor the GGUF has one. Use DeepSeek's format explicitly:
   <｜User｜>your question here<｜Assistant｜>
2. CUDA offload used to abort in ggml_cuda_op_concat (GGML_ASSERT(src0->type == GGML_TYPE_F32)) because the backend advertised concat support for types its kernel can't handle. Fixed in bati.cpp — the CUDA backend now reports concat support only for F32, so those nodes fall back to CPU instead of crashing. Use bati.cpp at or after that fix, or run CPU/Metal. (Apple Metal users were never affected.)

✨ What BatiAI did

• Quantized from the official DeepSeek weights — never a re‑quant of someone else's GGUF
• Handled the new UE8M0 / INT8‑expert storage format end to end
• Verified in Korean — most GGUF publishers never check this
• Fixed the runtime bug we found along the way, in the open, in bati.cpp
• BatiAI metadata‑signed

📜 License — MIT