SuperDeepseek-V4-Flash-abliterated
A less-refusing DeepSeek V4 Flash full checkpoint with verified 1M-token retrieval, intact tool use, and 123 tok/s-class two-node decode.
SuperDeepseek-V4-Flash-abliterated is the general, independently downloadable full-checkpoint release of SuperDeepseek. It is built from
deepseek-ai/DeepSeek-V4-Flash-0731, keeps DeepSeek's official native tensor layout, and fuses the selected OBLITERATUS + SuperTune update directly into the checkpoint. No LoRA or inference-time adapter is required.
Why this release
- Far fewer unnecessary refusals: worst-mode refusal fell from 97.92% to 4.17% in the paired gate.
- Tools stayed intact: tool compliance and correct-tool selection both remained 100%.
- Real 1M-context proof: a 1,028,621-token prompt was accepted and its hidden needle was retrieved.
- Fast verified serving: the released weights reached 123.35 aggregate tok/s on forced decode and 123.28 aggregate tok/s on structured tool generation.
- Surgical weight edit: all 256 routed experts, routers, embeddings, mHC tensors, and untargeted parent tensors preserve their official representation.
Release at a glance
| |
|---|
| Architecture | DeepSeek V4 Flash, 304B-class MoE, 43 backbone + 3 MTP layers, 256 routed experts, top-6 |
| Checkpoint | Complete Transformers/safetensors checkpoint, about 169.5 GB, 48 parent shards plus fused SuperDeepseek overlays |
| Native precision | FP4 experts + FP8 E4M3 block-quantized paths + BF16 quality-sensitive paths |
| Targeted update | 46 attn.wo_b weight/scale pairs plus one bounded BF16 output-head recovery |
| Context | 1,048,576 configured; 1,028,621 actual prompt tokens accepted and retrieved |
| Decode | 123.3459 forced / 123.2819 structured-tool aggregate tok/s at p256/C6 |
| Behavior | Worst-mode refusal 97.92% -> 4.17%; tool gates 100% |
Checkpoint format and precision
This is the native-format full checkpoint, not a full-BF16 reconstruction. DeepSeek's official V4 Flash checkpoint is itself hybrid precision; no authentic all-BF16 upstream checkpoint is published. This release deliberately preserves that official representation instead of dequantizing it and presenting approximated BF16 weights as “original.”
| Component | Precision / storage |
|---|
| MoE expert weights | FP4, inherited from the official expert_dtype=fp4 layout |
| Block-quantized paths | FP8 E4M3, dynamic activation scaling, 128x128 blocks, UE8M0 scales |
43 backbone + 3 MTP attn.wo_b updates | Deterministic FP8 weight/scale overlay, 92 tensors |
| Default unquantized and quality-sensitive paths | BF16 (torch_dtype=bfloat16), with upstream F32 metadata/normalization where defined |
| Output-head recovery | One bounded BF16 head.weight overlay, rank 64, relative Frobenius delta 0.0025 |
| Verified serving KV cache | NVFP4 DS-MLA |
The parent is pinned to deepseek-ai/DeepSeek-V4-Flash-0731@9e165c30e2704aec5d9d593cce3eebd58bbef1cb. All 48 parent shard identities were checked against that revision.
For the release name and scripts tuned specifically for a two-node DGX Spark deployment, see
Jiunsong/SuperDeepseek-V4-Flash-abliterated-MQ-2xDGX. Both repositories contain the same released SuperDeepseek weights; this repository is the general/native-layout edition, while the sibling makes the measured MQ + 2xDGX deployment target explicit in its name.
What changed
The release uses two measured refusal-subspace passes:
- A robust rank-1 direction fitted across chat, think-high, and think-max modes, applied at strength 2.
- A second rank-1 residual direction recaptured from the baked first pass, orthogonalized against it, and applied at strength 0.5.
- A bounded rank-64 output-head recovery with relative Frobenius delta 0.0025.
The final checkpoint changes only the 43 backbone and three MTP attn.wo_b weight/scale pairs plus the bounded output head. The routed experts remain untouched.
Behavior and capability
| Metric | Official parent | SuperDeepseek |
|---|
| Worst-mode refusal | 97.92% | 4.17% |
| Worst empty answer | 0.00% | 0.00% |
| Worst tool compliance | 100.00% | 100.00% |
| Worst correct-tool rate | 100.00% | 100.00% |
| Minimum capability mean | 0.9375 | 0.9583 |
The independently reloaded checkpoint reproduced the selected candidate's deterministic validation behavior. The release gate covered empty output, Unicode, repetition, serialization, reasoning, code, formatting, tool selection, and structured tool calls.
Measured performance
These are
aggregate concurrent post-first-token decode rates, not single-stream numbers. Six distinct prompts, fixed-length generation, and the sealed
sparkDash-style p256/C6 contract were used.
| Workload | Prompt / concurrency | Three trials | Median |
|---|
| Forced output | p256 / C6 | 117.7944, 123.3459, 125.9469 tok/s | 123.3459 tok/s |
| Structured tool output | p256 / C6 | 123.6285, 123.2819, 117.1843 tok/s | 123.2819 tok/s |
The benchmark follows sparkDash commit dfde4214f32b174880832a4d317d3c0567750ac5. Hardware and runtime were 2x NVIDIA DGX Spark / GB10, TP=2, direct CX-7 RoCEv2, vLLM, FlashInfer b12x MoE, DSpark K=1 speculative decoding, and NVFP4 DS-MLA KV cache. Hardware-specific figures should not be assumed on other systems.
Verified long context
| Actual prompt tokens | Accepted | Needle retrieved |
|---|
| 149,845 | Yes | Yes |
| 1,028,621 | Yes | Yes |
The configured maximum is 1,048,576 tokens. These are end-to-end acceptance and retrieval probes, not a claim of perfect recall for every task or needle position.
Two-node reference serving
The verified profile uses TP=2 over direct CX-7 RoCEv2 with the ghcr.io/anemll/dspark-vllm-gx10:0.1.1 runtime:
1vllm serve /model \
2 --served-model-name SuperDeepseek-V4-Flash-abliterated \
3 --tensor-parallel-size 2 \
4 --max-model-len 1048576 \
5 --kv-cache-dtype nvfp4_ds_mla \
6 --moe-backend flashinfer_b12x \
7 --enable-prefix-caching \
8 --async-scheduling \
9 --enable-chunked-prefill \
10 --speculative-config '{"method":"dspark","num_speculative_tokens":1,"draft_sample_method":"greedy"}'
The exact two-rank launcher and machine-readable release evidence are included under repro/ and evidence/.
Integrity
- Parent revision:
9e165c30e2704aec5d9d593cce3eebd58bbef1cb
- 48 parent shard names and hashes verified
- 92 FP8 overlay tensors: exactly 46
attn.wo_b weight/scale pairs
- One bounded BF16 output-head tensor
- Independent reload, paired validation, decode, reasoning, tool, long-context, and output-integrity gates passed
| Artifact | SHA-256 |
|---|
| SuperDeepseek FP8 overlay | 86c8494d0b02a01ccb4d4de5ad48a66d26f56341e1dd3eb17eb61b40710f73ba |
| BF16 head-recovery overlay | 3d49e3e05ba864666054328f475058a9558603aac70d1a191b352be65bce1428 |
| Sealed throughput report | 1cd7734205f4d03cddf241d5ed2a5f423f402d194222c9abc3bdacc965cce370 |
| 1M-context evidence | 6745453ee65b9e59581d715bf668bc710b0bc680fe726763e9a750dd25f45565 |
Responsible use and limitations
“Abliterated” describes a measured reduction of the selected refusal subspace. It does not make every answer correct, remove the need for deployment controls, or transfer responsibility away from the operator. The capability suites are finite regression gates, and the upstream model's license and limitations continue to apply.
License
MIT, following the upstream DeepSeek V4 Flash release.
