This model represents a linear merge of three distinct Llama 3-8b models using the Mergekit tool. The primary goal of this merge is to leverage the unique strengths of each base model, such as multilingual capabilities and specialized domain knowledge, into a more versatile and generalized language model.
By merging these models linearly, we combine their expertise into a unified model that performs well across various tasks, such as text generation, multilingual understanding, and domain-specific tasks.
-
Models Used:
- Danielbrdz/Barcenas-Llama3-8b-ORPO
- DeepMount00/Llama-3-8b-Ita
- lightblue/suzume-llama-3-8B-multilingual
-
Merging Tool: Mergekit
-
Merge Method: Linear merge with equal weighting (1.0) for all models
-
Tokenizer Source: Union
-
Data Type: float16 (FP16) precision
-
License: MIT License
-
Languages Supported: Multilingual, including English, Italian, and potentially others from the multilingual base models
1models:
2 - model: Danielbrdz/Barcenas-Llama3-8b-ORPO
3 parameters:
4 weight: 1.0
5 - model: DeepMount00/Llama-3-8b-Ita
6 parameters:
7 weight: 1.0
8 - model: lightblue/suzume-llama-3-8B-multilingual
9 parameters:
10 weight: 1.0
11merge_method: linear
12tokenizer_source: union
13dtype: float16
