This is a merge of pre-trained language models created using
mergekit.
-
Base Model: The foundation of this model is based on "refuelai/Llama-3-Refueled," which itself is a refined version of the Llama3-8B model, renowned for its instruction-following capabilities and adaptability across various domains.
-
Merged Models:
- ruslanmv/ai-medical-model-32bit: A model fine-tuned specifically for answering technical medical questions, providing a solid base of medical knowledge.
- Locutusque/Llama-3-Hercules-5.0-8B: Known for its ability to follow complex instructions and handle conversational interactions effectively, especially in scientific and technical contexts.
This model was merged using the
DARE TIES merge method using
refuelai/Llama-3-Refueled as a base.
1
2
3models:
4 - model: Locutusque/Llama-3-Hercules-5.0-8B
5 parameters:
6 weight: [0.25, 0.45, 0.35, 0.45, 0.25]
7 density: [0.1, 0.25, 0.5, 0.25, 0.1]
8 - model: refuelai/Llama-3-Refueled
9 - model: ruslanmv/ai-medical-model-32bit
10 parameters:
11 weight: [0.55, 0.45, 0.25, 0.45, 0.55]
12 density: [0.1, 0.25, 0.5, 0.25, 0.1]
13merge_method: dare_ties
14base_model: refuelai/Llama-3-Refueled
15parameters:
16 int8_mask: true
17dtype: bfloat16
18
