models:
    - model: jeonsworld/CarbonVillain-en-10.7B-v4
      parameters:
        weight: 1.0
    - model: vicgalle/NeuralBeagle-11B
      parameters:
        weight: 0.5
merge_method: linear

dtype: float16

Evaluations

At the time of its creation (21-01-2024), it is the best model in the Open LLM Leaderboard for its size class (10.7B-11B), and also 13B models:

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	74.64
AI2 Reasoning Challenge (25-Shot)	71.84
HellaSwag (10-Shot)	88.93
MMLU (5-Shot)	66.62
TruthfulQA (0-shot)	69.43
Winogrande (5-shot)	84.06
GSM8k (5-shot)	66.94

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	22.36
IFEval (0-Shot)	54.15
BBH (3-Shot)	33.06
MATH Lvl 5 (4-Shot)	5.51
GPQA (0-shot)	6.94
MuSR (0-shot)	9.19
MMLU-PRO (5-shot)	25.29

Notice

Part of the model is derived from CC-BY-NC-SA-licensed work
Hence, the model cannot be used for commercial purposes
And redistribution must be under the same license (CC-BY-NC-SA 4.0)

vicgalle/CarbonBeagle-11B

CarbonBeagle-11B

Merge Details

Merge Method

Models Merged

Configuration

Evaluations

Open LLM Leaderboard Evaluation Results

Open LLM Leaderboard Evaluation Results

Notice

vicgalle/
CarbonBeagle-11B