This is Mistral-10.7B-v0.2, a depth-upscaled version of
alpindale/Mistral-7B-v0.2-hf.
This model is intended to be used as a basis for further fine-tuning, or as a drop-in upgrade from the original 7 billion parameter model.
This is a merge of pre-trained language models created using
mergekit.
This model was merged using the passthrough merge method.
1dtype: bfloat16
2merge_method: passthrough
3slices:
4- sources:
5 - layer_range: [0, 24]
6 model: /Users/jsarnecki/opt/Workspace/alpindale/Mistral-7B-v0.2-hf
7- sources:
8 - layer_range: [8, 32]
9 model: /Users/jsarnecki/opt/Workspace/alpindale/Mistral-7B-v0.2-hf
10
