Views
No views yet
deepseek-ai/DeepSeek-V4-Flash — the
April 2026 preview release that DeepSeek-V4-Flash-0731 officially
superseded ("same model architecture and size … only re-post-trained"). The
Jacobian lens (Anthropic, Verbalizable Representations Form a Global Workspace
in Language Models) reads out what an internal residual is disposed to make
the model say:lens_l(h) = unembed( J_l @ h ), J_l = E[ ∂h_target / ∂h_l ]J_l is a
closed-form running mean of per-prompt Jacobians.hc_mult = 4 parallel residual streams per position
(hyper-connections), so a square J_l ∈ R^{d×d} is not well-defined. These
lenses are the rectangular generalization: source is the four flattened
streams (16384), target is the hc_head stream-collapse output (4096), so the
collapse operator is absorbed into J_l.| file | prompts | layers | shape | note |
|---|---|---|---|---|
lens.pt | n=100 | L19–39 (21 layers) | [4096, 16384] fp16 | recipe matched to the 0731 lens_n100.pt |
skip_first=16, per-prompt normalization, and dim_batch are all
identical to the 0731 n=100 lens, so any preview-vs-0731 lens difference is
attributable to the post-training change rather than to fitting settings. The
chat encodings of the two checkpoints were verified byte-identical for the
modes used before any comparison was made.jlens package:1import transformers, jlens
2
3name = "deepseek-ai/DeepSeek-V4-Flash"
4hf = transformers.AutoModelForCausalLM.from_pretrained(name, dtype="auto", device_map="auto")
5tok = transformers.AutoTokenizer.from_pretrained(name)
6model = jlens.from_hf(hf, tok)
7
8lens = jlens.JacobianLens.from_pretrained("xiangchensong/jacobian-lens-deepseek-v4-flash-preview")