You can deploy this model and start earning money today!

RichardErkhov/
Joseph717171_-_Mistral-12.25B-v0.2-gguf

Name: Joseph717171_-_Mistral-12.25B-v0.2-gguf
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Mistral-12.25B-v0.2 - GGUF

Model creator: https://huggingface.co/Joseph717171/
Original model: https://huggingface.co/Joseph717171/Mistral-12.25B-v0.2/

Name	Quant method	Size
Mistral-12.25B-v0.2.Q2_K.gguf	Q2_K	4.33GB
Mistral-12.25B-v0.2.IQ3_XS.gguf	IQ3_XS	4.81GB
Mistral-12.25B-v0.2.IQ3_S.gguf	IQ3_S	5.07GB
Mistral-12.25B-v0.2.Q3_K_S.gguf	Q3_K_S	5.04GB
Mistral-12.25B-v0.2.IQ3_M.gguf	IQ3_M	5.24GB
Mistral-12.25B-v0.2.Q3_K.gguf	Q3_K	5.62GB
Mistral-12.25B-v0.2.Q3_K_M.gguf	Q3_K_M	5.62GB
Mistral-12.25B-v0.2.Q3_K_L.gguf	Q3_K_L	6.11GB
Mistral-12.25B-v0.2.IQ4_XS.gguf	IQ4_XS	6.3GB
Mistral-12.25B-v0.2.Q4_0.gguf	Q4_0	6.57GB
Mistral-12.25B-v0.2.IQ4_NL.gguf	IQ4_NL	6.64GB
Mistral-12.25B-v0.2.Q4_K_S.gguf	Q4_K_S	6.62GB
Mistral-12.25B-v0.2.Q4_K.gguf	Q4_K	6.99GB
Mistral-12.25B-v0.2.Q4_K_M.gguf	Q4_K_M	6.99GB
Mistral-12.25B-v0.2.Q4_1.gguf	Q4_1	7.29GB
Mistral-12.25B-v0.2.Q5_0.gguf	Q5_0	8.01GB
Mistral-12.25B-v0.2.Q5_K_S.gguf	Q5_K_S	8.01GB
Mistral-12.25B-v0.2.Q5_K.gguf	Q5_K	8.22GB
Mistral-12.25B-v0.2.Q5_K_M.gguf	Q5_K_M	8.22GB
Mistral-12.25B-v0.2.Q5_1.gguf	Q5_1	8.73GB
Mistral-12.25B-v0.2.Q6_K.gguf	Q6_K	9.53GB
Mistral-12.25B-v0.2.Q8_0.gguf	Q8_0	12.35GB

Original model description:

base_model: [] library_name: transformers tags:

mergekit
merge license: apache-2.0

Credit for the model card's description goes to ddh0 and mergekit

Credit for access and conversion of Mistral-7B-v0.2 goes to alpindale (from MistalAI's weights to HF Transformers)

Mistral-12.25B-v0.2

This is Mistral-12.25B-v0.2, a depth-upscaled version of alpindale/Mistral-7B-v0.2-hf.

This model is intended to be used as a basis for further fine-tuning, or as a drop-in upgrade from the original 7 billion parameter model.

Paper detailing how Depth-Up Scaling works: SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scaling

This is a merge of pre-trained language models created using mergekit.

UpStage's conclusionary limitations of their research:

"Our study on the Depth Up-Scaling (DUS) has important limitations and considerations. One key limitation is the need for more thorough explorations of hyperparameters used in the DUS approach. Namely, we removed m = 8 layers from both ends of our base model, primarily due to hardware limitations. However, we have not yet determined if this value is optimal for enhancing performance. The extended time and cost of continued pretraining made it challenging to conduct more comprehensive experiments, which we aim to address in future work through various comparative analyses."

This model was made to help test whether 10.7B parameters (m = 8) is better or worse than m < 8 (10.7B+ parameters)

Merge Details

Merge Method

This model was merged using the passthrough merge method.

Models Merged

The following models were included in the merge:

/Users/jsarnecki/opt/Workspace/alpindale/Mistral-7B-v0.2-hf

Configuration

The following YAML configuration was used to produce this model:

yaml

dtype: bfloat16
merge_method: passthrough
# Depth UpScaled (DUS) version of Mistral-7B-v0.2
# where m = 4 (The number of layers to remove from the model)
#       s = 56 (The number of layers the model will have after the DUS)
slices:
- sources:
  - layer_range: [0, 28]
    model: /Users/jsarnecki/opt/Workspace/alpindale/Mistral-7B-v0.2-hf
- sources:
  - layer_range: [4, 32]
    model: /Users/jsarnecki/opt/Workspace/alpindale/Mistral-7B-v0.2-hf 

exllama (Thanks to blockblockblock)

https:// huggingface.co/blockblockblock/ Mistral-12.25B-Instruct-vO.2-bpw2.5 https://huggingtace.co/blockblockblock/ Mistral-12.25B-Instruct-v0.2-bpw3 https://huggingface.co/blockblockblock/ Mistral-12.25B-Instruct-vO.2-bpw3.5 https://huggingface.co/blockblockblock/ Mistral-12.25B-Instruct-vO.2-bpw3.7 https://huggingface.co/blockblockblock/ Mistral-12.25B-Instruct-vO.2-bpw4 https://huggingface.co/blockblockblock/ Mistral-12.25B-Instruct-v0.2-bpw4.2 https://huggingface.co/blockblockblock/ Mistral-12.25B-Instruct-v0.2-bpw4.4 https://huggingface.co/blockblockblock/ Mistral-12.25B-Instruct-v0.2-bpw4.6 https://huggingface.co/blockblockblock/ Mistral-12.25B-Instruct-vO.2-bpw4.8 https://huggingface.co/blockblockblock/ Mistral-12.25B-Instruct-v0.2-bpw5 https://huggingface.co/blockblockblock/ Mistral-12.25B-Instruct-vO.2-bpw5.5 https://huggingface.co/blockblockblock/ Mistral-12.25B-Instruct-v0.2-bpw6

RichardErkhov/Joseph717171_-_Mistral-12.25B-v0.2-gguf