This is a merge of pre-trained language models created using
mergekit.
This model was merged using the
TIES merge method using
OdiaGenAI/mistral_hindi_7b_base_v1 as a base.
1models:
2 - model: OdiaGenAI/mistral_hindi_7b_base_v1
3 # no parameters necessary for base model
4 - model: BioMistral/BioMistral-7B
5 parameters:
6 density: 0.7
7 weight: [0.1, 0.3, 0.6, 0.7] # weight gradient
8merge_method: ties
9base_model: OdiaGenAI/mistral_hindi_7b_base_v1
10parameters:
11 normalize: true
12 int8_mask: true
13dtype: bfloat16
