This study proposes a novel and straightforward approach to unlocking the potential of top-ranking Large Language Models (LLMs) from the Open LLM Leaderboard, leveraging the DARE (Drop and REscale) technique. DARE facilitates efficient model merging by minimizing delta parameter redundancy from fine-tuned models. We integrate a top-performing multilingual LLM with a specialized Korean language model using DARE. The merged model is evaluated on six benchmark tasks and MT-Bench, focusing on reasoning capabilities. Results show that incorporating the Korean language model achieves a significant performance improvement of 1.69% on average across the six benchmark tasks, and notably demonstrates over 20% higher performance on GSM8K, which requires complex reasoning skills. This suggests that the inherent complexity and rich linguistic features of the Korean language contribute to enhancing LLM reasoning abilities. Moreover, the model exhibits superior performance on MT-Bench, demonstrating its effectiveness in real-world reasoning tasks. This study highlights DARE's potential as an effective method for integrating specialized language models, demonstrating the ability to unlock existing language models' potential for advanced tasks.

Paper : http://dx.doi.org/10.2139/ssrn.5063211

Merging the Korean model leads to performance changes depending on the density value (d). The highest average score (76.23) is achieved with a density value of 0.25, representing a 1.70% improvement over the base model (Base-LM). Performance generally improves as the density value decreases from 0.4 to 0.25 but tends to decline when the value drops below 0.25. This suggests an optimal balance exists between preserving the knowledge of the original model and incorporating knowledge from the Korean model. Therefore, we use a density value of 0.25 for subsequent experiments when merging with the Korean language model. Interestingly, despite Ko-LM's very low GSM8K score of 30.86, merging with Ko-LM still results in a performance improvement of over 20%.

Detailed results can be found here

This model was merged using the DARE TIES merge method using swap-uniba/LLaMAntino-3-ANITA-8B-Inst-DPO-ITA as a base.

The following models were included in the merge:

The following YAML configuration was used to produce this model: