DeepSeek-V4-Flash-0731 — Abliterated

⚠️ Uncensored model. Safety refusals have been substantially removed. You are responsible for how you use it. Intended for local/research use on hardware you control. It will attempt almost any request.

Highlights

Validation (before → after abliteration)

How it was built

1. Refusal directions. Forward hooks on all 43 attn.wo_b modules of a live vLLM TP=2 deployment captured last-token sublayer outputs over 256 AdvBench harmful vs 256 Alpaca harmless prompts, in all three reasoning modes (chat, think-high, think-max), giving per-layer difference-of-means directions r_ℓ ∈ ℝ⁴⁰⁹⁶. Cross-rank digests were asserted identical on every prompt, since wo_b is RowParallelLinear and a pre-all-reduce hook would silently capture a half-direction. Stability: median split-half cosine 0.9863, median held-out AUC 0.9976, all 43 layers above 0.90.
2. Merge + bake. The three per-mode direction sets were merged and applied as W ← W − λ·r̂(r̂ᵀW) at λ = 2.5, in float64, directly on the FP8 e4m3 weights: dequantize with the ue8m0 128×128 block scales, project, then re-quantize holding the original block exponents fixed so every element the projection did not move re-encodes to its identical original byte. Overflow is clamped rather than rescaled (6,877 of 94,208 blocks overflowed; 7,821 of 1.54 B elements clamped; max overshoot 1.186×). The 3 DSpark stages inherit the deepest backbone direction — an unedited drafter would propose refusal tokens the edited verifier rejects, collapsing acceptance.

Serving (vLLM, 2× DGX Spark)

• --moe-backend flashinfer_b12x is not optional. auto resolves to DEEPGEMM_MXFP4, the slow path on sm_121. VLLM_USE_B12X_MOE=1 alone does not override it — the CLI flag does.
• --linear-backend must stay auto; flashinfer_b12x has no kernel for V4's FP8 block-scaled linears and hard-fails at load.
• NCCL_IB_HCA must name a device that exists. GB10 renames its ConnectX-7 RDMA devices to rocep*/roceP2p* — there is no mlx5_N. NCCL_IB_HCA is a prefix filter: a non-matching value makes NCCL log "no device found" and silently fall back to TCP sockets, costing ~2.2× decode throughput (12 → 27 tok/s here). Verify with NCCL_DEBUG=INFO | grep 'NET/IB'.
• num_speculative_tokens must be ≤ 5 (dspark_block_size). The model card's 7 boots unvalidated on vLLM 0.25.x and then misbehaves — the drafter emits exactly block_size tokens per pass and multi-block drafting is unimplemented.
• --max-num-batched-tokens ≥ 8192 + max_num_seqs × (k−1), else vLLM silently degrades the scheduler budget.
• Prefix caching is safe here only because VLLM_DSPARK_GPU_REJECTED_CONTEXT_MASK=1 is set (vLLM #47930 otherwise collapses draft acceptance to ~0.5 %). Measured with it on: acceptance 48.7 %, unchanged.
• nvidia-smi reports memory as N/A on GB10 and torch.cuda.mem_get_info is misleading on unified memory — use free -h.

Notes for agent frameworks

• Reasoning modes: chat, think-high, think-max. There is no HuggingFace chat template — prompts go through the repo's encoding/encoding_dsv4.py (encode_messages(messages, thinking_mode, reasoning_effort)). Calling tokenizer.apply_chat_template silently falls back to a generic format instead of raising.
• Enabling thinking over the OpenAI API: chat_template_kwargs: {"thinking": true, "reasoning_effort": "max"}. A top-level reasoning_effort field is ignored — vLLM's deepseek_v4 parser gates on chat_template_kwargs["thinking"] AND reasoning_effort != "none". Reasoning is returned in the reasoning field, separate from content.
• Tool calls use DeepSeek's native DSML block, not JSON. parse_message_from_completion_text raises on malformed output, which makes it a clean compliance signal.
• Benchmarking: read usage.completion_tokens, never count SSE deltas. Under speculative decoding vLLM emits one chunk per decode step carrying every accepted token, so counting chunks measures steps/s and under-reports by the acceptance length (~3.4× here).

Findings

• Refusal is encoded along two substantially different directions. Chat-mode and thinking-mode directions have median cosine 0.46 on the 36 layers where both separate at AUC > 0.99, and they diverge with depth (0.88 at layer 0, −0.08 at layer 42). The two thinking modes agree with each other (0.90). A direction fitted only on non-thinking prompts leaves thinking-mode refusal substantially intact — hence the merged direction here.
• mHC does not block abliteration. V4 replaces ordinary residuals with hyper-connections, so the carrier is [B,S,4,4096] and output_hidden_states returns that carrier rather than a hidden state. But the mHC write is rank-1 across streams — hc_post broadcasts one 4096-vector into all four with per-token scalars — so removing a direction at the sublayer output removes it from every stream at once. Measured: split-half cosine 0.9863, held-out AUC 0.9976, in every reasoning mode.
• A baked FP8 edit is structurally weaker than a runtime hook. A rank-1 edit spread over 4096 dims perturbs each element by ~1/√4096 of the row's component — measured at 0.253× the e4m3 quantization step — so most elements round back to their original byte. Best achievable direction removal on a real tensor is ~68 %, minimised near λ≈1.5. This is why λ must be calibrated against the baked model rather than inherited from a search run, and plausibly why published work converges on over-projection at λ≈2.5 rather than a "clean" λ=1.

Limitations

• Capability was gated on tool-calling only. MMLU-Pro, GSM8K, HumanEval and similar benchmarks were not run. The claim is "tool-calling and refusal are measured"; it is not "general capability is unchanged". The published precedent at this operating point reports capability flat-to-positive, which is reassuring but is not a measurement of this checkpoint.
• Long-context behaviour is unvalidated. Directions were captured from ~60-token prompts. CSA's Lightning Indexer selects a completely different token set at 168k than at 60 tokens, so it is an open question whether refusal suppression holds at depth. No published DeepSeek-V4 abliteration has been validated past 32k either. Needle-in-haystack and refusal-at-depth were not run here.
• Refusal is scored by marker matching on the parsed answer content (thinking block excluded), with long generations to catch V4's delayed-refusal pivot. It is not an LLM-judge evaluation, and it will miss refusals phrased in ways the marker set does not cover.
• Two categories resist attention-only editing. Published work on this architecture reports PII-doxing and self-harm prompts retaining materially higher refusal than the aggregate at the same operating point. Not separately measured here.
• Sample sizes are small (n=48 chat, n=24 per thinking mode). Zero refusals observed is not zero refusal rate — see the Wilson bounds above.

Files

License & credits