Question	Answer
What is it for?	Studying a target-only MLX conversion of DeepSeek V4 Flash 0731 and its measured limitations on Apple Silicon.
What did I contribute?	Philip John Basile's target-only conversion, companion-runtime integration, and comparison evidence. The original model is by DeepSeek.
What runs it?	The companion native DeepSeek-V4 `mlx-serve` path or the specifically pinned experimental oMLX/MTPLX integrations documented below. General compatibility with current app releases is not established by these historical tests.
What was tested?	An M5 Max with 128 GB unified memory; recorded load/generation checks and the August 2026 task/GPQA comparisons below.
How much memory?	About 103.85 GB of tensor payload; the recorded native smoke peaked at 100.181 GB. Only the 128 GB M5 Max configuration was tested.
What are the limits?	No MTP drafter, no vision, substantial measured quality/response-format regressions. The card records a faster GGUF alternative on the tested machine.

Quick start — recorded native smoke

After downloading the artifact and building the companion DeepSeek-V4 runtime described below, use:

bash

mlx-serve \
  --model /path/to/DeepSeek-V4-Flash-0731-MLX-M5Max-TargetOnly \
  --prompt 'Reply with exactly: READY' \
  --max-tokens 4 --temp 0 \
  --no-pld --no-decode-attn-quant --no-vision \
  --ctx-size 512 --timeout 300

This is the historical validated command shape, not a claim that an arbitrary mlx-serve installation supports the artifact. Full source/runtime pins and adverse results follow.

Status (2026-08-12): this MLX artifact is superseded on the M5 Max by the DwarfStar (antirez/ds4) GGUF build of the same model. On this hardware we measured the DwarfStar build at roughly 2–3× faster decode with strictly better fidelity (0.39 avg token-NLL vs official 0731 continuations, 86% top-1, 78% top-N recall, 98.7% ranking agreement, via DwarfStar's own score_official). This MLX artifact is retained as an experimental, honest reference for the MLX/mlx-serve path — its measured limits are stated below, not hidden — and as provenance evidence for the conversion work.

This is an experimental, target-only MLX derivative of deepseek-ai/DeepSeek-V4-Flash-0731, pinned to upstream revision 7872f01b1d1fe23eabc4c98b48bffcef5a386062.

It is intended for public testing on a 128 GB Apple Silicon Mac. Real M5 Max load-and-generation smokes passed, as did a full 198-question GPQA Diamond comparison against the OpenRouter reference (results below). The direct API arm still has a substantial response-format regression; a separate choice-constrained diagnostic tracks the reference much more closely. The artifact is also slower than the DwarfStar GGUF build on the same hardware. Do not interpret this upload as a general no-quality-loss or no-speed-loss claim; prefer the DwarfStar build on this machine.

What is included

43 target-model layers in 44 safetensor shards
2,320 target tensors
103,848,946,780 tensor-payload bytes
103,855,768,335 total logical bytes in the validated local view
serial target model only: num_nextn_predict_layers=0
no MTP/DSpark drafter weights

Quantization recipe:

expert w1/w3, layers 0–38: Q2 group 128
expert w2, layers 0–38: Q3 group 128
expert w1/w2/w3, layers 39–42: Q4 group 64
attention, shared-expert, embedding, and head projections: affine Q8 group 64

Verified hardware smoke

Observed on an Apple M5 Max with 128 GB unified memory using the companion ReleaseFast mlx-serve DeepSeek-V4 runtime:

all 2,320 tensors from all 44 shards loaded
model reached Model ready
deterministic prompt output: exactly READY
first-touch prompt: 10 tokens at 0.937 tokens/s
decode: 2 tokens at 20.326 tokens/s
peak memory: 100.181 GB

This two-token decode is a smoke result, not a throughput benchmark. Only a 128 GB M5 Max has been tested; smaller-memory Macs are not claimed supported.

Original-model comparison

50-task quality benchmark (primary)

A deterministic 50-task suite (8 exact-copy, 8 JSON, 12 arithmetic, 10 multi-step reasoning, 6 code, 3 multilingual, 3 classification) is run with objective programmatic graders on all three arms, 3 draws per task, pass = majority of draws, temperature 0, non-streaming, logprobs enabled. The original is served via OpenRouter's pinned CoreWeave FP8 endpoint (coreweave/fp8, no fallbacks), matching the published four-case reference.

Arm	Graded	Pass	Rate	mean tok/s
Original DeepSeek-V4-Flash-0731 (CoreWeave FP8)	50	50	100%	n/a (network-inclusive)
Target-only, direct mlx-serve	50	37	74%	10.9
Target-only, MTPLX-routed	50	37	74%	10.2

By category (direct mlx-serve; MTPLX identical):

Category	Original	Target-only	Verdict
exact copy	8/8	8/8	parity
JSON	8/8	8/8	parity
math arithmetic	12/12	7/12	regression
multi-step reasoning	10/10	3/10	regression
code	6/6	6/6	parity
multilingual	3/3	3/3	parity
classification	3/3	2/3	regression

This is a real, measured quality gap. The 8-bit affine target-only artifact matches the original on copy/JSON/code/multilingual but regresses on arithmetic and multi-step reasoning (observed: 17×23 → 289 not 391; Tom's 3-1 apples doubled → 5 not 4; vowels in "hello" → 4 not 2). This comparison does not establish speed parity: local decode throughput was about 10–11 tok/s, while remote timing includes network overhead and is not a comparable engine-speed measurement. The direct and MTPLX paths are byte-identical (same delegated engine).

Receipt: receipts/dsv4/2026-08-12-50task-three-arm-comparison.json; per-run receipts receipts/dsv4/quality-speed-20260812T030200Z.json (remote 50/50) and quality-speed-20260812T034809Z.json (local 37/50).

Full GPQA Diamond comparison (198 questions)

On 2026-08-13 we ran all 198 GPQA Diamond questions with one epoch, seed 42, temperature 0, top-p 1, a 32-token output limit, and chain of thought/reasoning disabled. OpenRouter was pinned to deepseek/deepseek-v4-flash-0731 on coreweave/fp8, with fallback disabled. The exact prompt bytes and shuffled answer order from that frozen reference run were then replayed locally through oMLX.

Arm	Correct	Accuracy	Format failures
OpenRouter CoreWeave FP8	101/198	51.0%	1
Target-only oMLX, direct output	70/198	35.4%	60
Target-only oMLX, choice-constrained diagnostic	97/198	49.0%	0

The direct-output arm is the strict API-behavior comparison. All 60 local format failures consumed the 32-token budget explaining the problem instead of emitting the requested ANSWER: X. Among the 138 local responses that did provide an answer, local accuracy was 70/138 (50.7%); OpenRouter scored 75/138 (54.3%) on that same subset.

The constrained arm used oMLX's existing decoder constraint ANSWER: A/B/C/D. It is a knowledge-retention diagnostic, not an apples-to-apples serving comparison. It scored 97/198 (49.0%), 2.0 percentage points below the 101/198 reference; the paired difference was not significant in this sample (exact McNemar two-sided p=0.659). This supports retained task knowledge, but it does not erase the real unconstrained instruction-following regression or establish general no-quality-loss parity.

All 198 OpenRouter generation IDs were independently checked in the provider ledger: CoreWeave only, canonical model deepseek/deepseek-v4-flash-20260731, zero cancellations, 197 stop and one retained length finish. The reference run used no HTTP retries and cost $0.00644646. Answer-letter mapping was also independently recomputed for all 198 shuffled samples against the immutable GPQA CSV: every archived target matched the presented correct choice, both local prompt hashes matched the reference prompt on all 198 samples, and all three stored score arms recomputed exactly (101, 70, and 97 correct). Aggregate methodology and receipts are in benchmarks/gpqa-diamond-20260813/.

Four-case behavioral gate (secondary, superseded by the above)

The same four public deterministic prompts against the same pinned CoreWeave FP8 endpoint produced byte-identical outputs on the two exact-copy cases, whitespace-equal JSON, and semantically-equivalent Spanish; receipt receipts/dsv4/2026-08-11-three-arm-20260812T024136Z.json. The original four-case remote run cost $0.00003432.

Broader public-task evaluation remains pending. In particular, this artifact does not claim the upstream Terminal Bench, NL2Repo, Cybergym, DeepSWE, Toolathlon, Agents' Last Exam, AutomationBench, or DSBench scores.

Running it

The validated baseline uses the companion native mlx-serve DeepSeek-V4 runtime with prompt lookup, lossy decode-attention quantization, and vision all disabled:

bash

mlx-serve \
  --model /path/to/DeepSeek-V4-Flash-0731-MLX-M5Max-TargetOnly \
  --prompt 'Reply with exactly: READY' \
  --max-tokens 4 --temp 0 \
  --no-pld --no-decode-attn-quant --no-vision \
  --ctx-size 512 --timeout 300

Experimental native oMLX support is available in PhilipJohnBasile/omlx@a34b0408 and is proposed upstream in jundot/omlx#2640. On the same M5 Max, that exact build loaded all 44 shards (96.83 GB active model memory), made the native DeepSeek indexer kernels available, returned the requested deterministic answers, completed the GPQA runs above, and cleanly released 96.71 GB on shutdown. This support is not yet in a released oMLX app.

Place the model inside an oMLX model-directory parent, then serve that parent:

bash

omlx-cli serve \
  --model-dir /path/to/parent-containing-the-model \
  --host 127.0.0.1 --port 8000 \
  --max-concurrent-requests 1 --memory-guard aggressive \
  --no-cache --no-hf-cache

Unsloth does not directly load or train this MLX artifact. The separate Unsloth integration currently provides pinned configuration/GGUF preflight and llama.cpp launch guidance only; it makes no Mac training claim.

Experimental MTPLX support is available via the codex/deepseek-v4-mlxserve-backend branch (backend release + gate/streaming fixes: 14413c2; backend initial release: 0bbe062e3a25a5de8cb31f3e8948c76516ff8404). It delegates to the companion native mlx-serve DeepSeek-V4 runtime and keeps this target-only artifact on the AR path. The backend sets MLX_SERVE_WIRED=fit for the child by default (override with MTPLX_DSV4_WIRED):

bash

mtplx pull philipjohnbasile/DeepSeek-V4-Flash-0731-MLX-M5Max-TargetOnly

MTPLX_MLX_SERVE_BIN=/path/to/mlx-serve \
mtplx serve \
  --model philipjohnbasile/DeepSeek-V4-Flash-0731-MLX-M5Max-TargetOnly \
  --no-mtp --host 127.0.0.1 --port 8000 --yes

Two short non-streaming MTPLX smokes completed at about 23.5 output tokens/s. An earlier streaming collapse (0.462 tokens/s then a zero-headroom refusal) was traced to the launch memory policy, not the model: with MLX_SERVE_WIRED=off the unwired 100 GB working set thrashes. Under the fit wired-residency policy (MLX_SERVE_WIRED=fit, MLX_SERVE_WIRED_SLACK_MB=0), the same server streams over HTTP at 28.3 decode tokens/s with no swap activity and 100.185 GB peak memory. Use that policy when serving:

bash

MLX_SERVE_WIRED=fit MLX_SERVE_WIRED_SLACK_MB=0 \
MTPLX_MLX_SERVE_BIN=/path/to/mlx-serve \
mtplx serve \
  --model philipjohnbasile/DeepSeek-V4-Flash-0731-MLX-M5Max-TargetOnly \
  --no-mtp --host 127.0.0.1 --port 8000 --yes

Representative multi-request stress and long-context streaming still warrant a dedicated load test before any throughput claim is published.

Provenance

local materialization SEAL SHA-256: 50cd20ae84b6c7ebe79c27e08da89c3e419fcde5b529fbd6c47f6387bbf0e79f
target-only input contract SHA-256: d69c6fa36d909d0bfc964324fac00e793798088bfffc71528d96c10cb45a4b3c
reviewed materializer SHA-256: 55d053b4daee2556aef30ea41993acde660c759a070e2af5156c7dd5af40d275
materializer tests SHA-256: 83f0c3916e95c02bfd623fd9c05343d2b87e9a9d781709730dddd31b75adcb53
tested ReleaseFast runtime binary SHA-256: 5c85c7cde0c4e1c1a9960e2ea1d9b48ada2c4f20f23b3c1617c9e9a5bb633d42 (includes the JSON logprobs UTF-8 fix: jsonEscape now escapes bytes that would otherwise form invalid UTF-8 inside a JSON string, so byte-level-BPE tokens that decode to a partial multi-byte sequence — e.g. the first two bytes of an ellipsis — no longer corrupt the response. This removes the intermittent UnicodeDecodeError that previously dropped a task from the benchmark receipt.)

The source model is licensed under MIT; see LICENSE and the upstream model card for attribution and its original terms.

Test feedback

When reporting a result, please include:

Mac model and unified-memory capacity
macOS version
runtime and model revision
exact flags and context length
whether the failure happened during load, prefill, or decode
peak memory and exact generated output