Views
No views yet
lemuralabs/DeepSeek-V4-Flash-5bit-mlx is an Apple-Silicon MLX quantization of deepseek-ai/DeepSeek-V4-Flash.| Property | Value |
|---|---|
| Base model | deepseek-ai/DeepSeek-V4-Flash |
| Architecture | DeepSeek-V4 Flash MoE, 284B total / 13B active, 1M context |
| Local profile | MLX-Affine-Q5 |
| Bundle size | 195.67 GB |
| Layout | Pre-stacked MLX switch_mlp layout |
| MTP head | Dropped |
| Validation | Safetensors header/index validation, metadata validation |
| Tensor class | Codec | Bits / handling |
|---|---|---|
| Linear/Embedding/SwitchLinear weights | MLX affine | 5-bit, group size 64 |
| Routed experts | MLX affine | pre-stacked switch_mlp tensors with .weight, .scales, .biases |
| Norms, router gate, mHC, sinks, APE, integer routing tables | passthrough | source precision preserved |
.weight.scales.biasespip install -U mlx-lm1from mlx_lm import load, generate
2
3model, tokenizer = load("lemuralabs/DeepSeek-V4-Flash-5bit-mlx")
4prompt = "Write a short note about MLX quantization."
5text = generate(model, tokenizer, prompt=prompt, verbose=True)
6print(text)encoding/ folder from the upstream repository.model-*.safetensors: standard MLX affine shardsmodel.safetensors.index.json: shard indexconfig.json, jang_config.json: MLX metadataencoding/: upstream DeepSeek-V4 prompt encoding referencemlx-community/*-4bit / *-8bit uploads and the local Lemura Labs/*-6bit-mlx style, while the README keeps the explicit recipe/validation structure used by larger DeepSeek-V4 quant uploads.