Views
No views yet
Mistral model preset.<s>[INST] {prompt} [/INST]| Name | Quant method | Size | Memory (RAM, vRAM) required (for full context of 32k tokens) |
|---|---|---|---|
| mistral-7b-instruct-v0.3.Q2_K.gguf | Q2_K | 2.72 GB | 6.78 GB |
| mistral-7b-instruct-v0.3.Q3_K_S.gguf | Q3_K_S | 3.17 GB | 7.19 GB |
| mistral-7b-instruct-v0.3.Q3_K_M.gguf | Q3_K_M | 3.52 GB | 7.52 GB |
| mistral-7b-instruct-v0.3.Q3_K_L.gguf | Q3_K_L | 3.83 GB | 7.80 GB |
| mistral-7b-instruct-v0.3.Q4_0.gguf | Q4_0 | 4.11 GB | 8.07 GB |
| mistral-7b-instruct-v0.3.Q4_K_S.gguf | Q4_K_S | 4.14 GB | 8.10 GB |
| mistral-7b-instruct-v0.3.Q4_K_M.gguf | Q4_K_M | 4.37 GB | 8.31 GB |
| mistral-7b-instruct-v0.3.Q4_K.gguf | Q4_K | 4.37 GB | 8.31 GB |
| mistral-7b-instruct-v0.3.Q4_1.gguf | Q4_1 | 4.56 GB | 8.48 GB |
| mistral-7b-instruct-v0.3.Q5_0.gguf | Q5_0 | 5.00 GB | 8.90 GB |
| mistral-7b-instruct-v0.3.Q5_K_S.gguf | Q5_K_S | 5.00 GB | 8.90 GB |
| mistral-7b-instruct-v0.3.Q5_K_M.gguf | Q5_K_M | 5.14 GB | 9.02 GB |
| mistral-7b-instruct-v0.3.Q5_K.gguf | Q5_K | 5.14 GB | 9.02 GB |
| mistral-7b-instruct-v0.3.Q5_1.gguf | Q5_1 | 5.45 GB | 9.31 GB |
| mistral-7b-instruct-v0.3.Q6_K.gguf | Q6_K | 5.95 GB | 9.78 GB |
| mistral-7b-instruct-v0.3.Q8_0.gguf | Q8_0 | 7.70 GB | 11.41 GB |
| mistral-7b-instruct-v0.3.f16.gguf | f16 | 14.50 GB | 17.74 GB |