You can deploy this model and start earning money today!

deepsweet/
GigaChat3.1-10B-A1.8B-MLX-oQ4

Name: GigaChat3.1-10B-A1.8B-MLX-oQ4
Availability: InStock

ai-sage/GigaChat3.1-10B-A1.8B

quantized

mit

4-bit

Views

No views yet

This model was converted to MLX format from ai-sage/GigaChat3.1-10B-A1.8B using oMLX v0.2.24 with oQ Quantization.

[!WARNING] Multi-Token Prediction (MTP) had to be disabled ("num_nextn_predict_layers": 0) and related layers had to be removed (model.layers.26.*).

Thanks RockTalk/GigaChat3.1-10B-A1.8B-MLX-4bit for the tip.

Settings:

Level: oQ4
Sensitivity model: none
Text Only: yes

deepsweet/GigaChat3.1-10B-A1.8B-MLX-oQ4

deepsweet/
GigaChat3.1-10B-A1.8B-MLX-oQ4