MLX mixed-precision quantized version of
lkevincc0/Step-3.5-Flash-REAP-128B-A11B for efficient local inference on Apple Silicon.
1from mlx_lm import load, generate
2
3model, tokenizer = load("shieldstackllc/Step-3.5-Flash-REAP-128B-A11B-mlx-mixed-4-6")
4response = generate(model, tokenizer, prompt="Hello!", verbose=True)
Or with
vMLX for native macOS inference.
Step-3.5-Flash is a large Mixture-of-Experts language model by
StepFun AI. This variant was pruned by
lkevincc0 using REAP (Router Expert Activation Pruning), reducing the expert count from the original to 173 while maintaining strong performance. The mixed-precision MLX quantization preserves higher fidelity on critical attention and feed-forward projections by using 6-bit for v_proj and down_proj layers.
This model was converted and optimized for
vMLX — a free, open source macOS native MLX inference engine for Apple Silicon. Download vMLX to run this model locally with zero configuration.
For questions, issues, or collaboration:
admin@vmlx.net
