TroyDoesAI_-_Codestral-21B-Pruned-gguf – AI Model by RichardErkhov | AlphaNeural AI

You can deploy this model and start earning money today!

RichardErkhov/
TroyDoesAI_-_Codestral-21B-Pruned-gguf

like

0

endpoints_compatible

Views

No views yet

Quantization made by Richard Erkhov.

Request more models

Codestral-21B-Pruned - GGUF

Model creator: https://huggingface.co/TroyDoesAI/
Original model: https://huggingface.co/TroyDoesAI/Codestral-21B-Pruned/

Name	Quant method	Size
Codestral-21B-Pruned.Q2_K.gguf	Q2_K	7.44GB
Codestral-21B-Pruned.IQ3_XS.gguf	IQ3_XS	8.24GB
Codestral-21B-Pruned.IQ3_S.gguf	IQ3_S	8.71GB
Codestral-21B-Pruned.Q3_K_S.gguf	Q3_K_S	8.67GB
Codestral-21B-Pruned.IQ3_M.gguf	IQ3_M	9.04GB
Codestral-21B-Pruned.Q3_K.gguf	Q3_K	9.67GB
Codestral-21B-Pruned.Q3_K_M.gguf	Q3_K_M	9.67GB
Codestral-21B-Pruned.Q3_K_L.gguf	Q3_K_L	10.54GB
Codestral-21B-Pruned.IQ4_XS.gguf	IQ4_XS	10.82GB
Codestral-21B-Pruned.Q4_0.gguf	Q4_0	11.3GB
Codestral-21B-Pruned.IQ4_NL.gguf	IQ4_NL	11.41GB
Codestral-21B-Pruned.Q4_K_S.gguf	Q4_K_S	11.37GB
Codestral-21B-Pruned.Q4_K.gguf	Q4_K	11.96GB
Codestral-21B-Pruned.Q4_K_M.gguf	Q4_K_M	11.96GB
Codestral-21B-Pruned.Q4_1.gguf	Q4_1	12.53GB
Codestral-21B-Pruned.Q5_0.gguf	Q5_0	13.77GB
Codestral-21B-Pruned.Q5_K_S.gguf	Q5_K_S	13.77GB
Codestral-21B-Pruned.Q5_K.gguf	Q5_K	14.12GB
Codestral-21B-Pruned.Q5_K_M.gguf	Q5_K_M	14.12GB
Codestral-21B-Pruned.Q5_1.gguf	Q5_1	15.01GB
Codestral-21B-Pruned.Q6_K.gguf	Q6_K	16.4GB
Codestral-21B-Pruned.Q8_0.gguf	Q8_0	21.24GB

Original model description:

license: apache-2.0 language:

For those trying to shoe horn this large model on your machine every GB of saved memory counts when offloading to System RAM!

Here is a pruned down the 22.2 Billion parameter model by 2 junk layers to make a 21.5B that doesnt appear to lose any sense of quality.

https://huggingface.co/mistralai/Codestral-22B-v0.1