* We found the zero-shot results are sensitive to chat templates and reported best achievable result for HumanInstruct for all models: we modified lm-evaluation-harness/lm_eval/tasks/humaneval/humaneval.yaml so that "```" can be considered as end of responses.
The merged models, along with the activation inner product matrices, are avaiable on the huggingface hub.
What's new?
RegMean solves a least square regression problem at each linear layer of the transformer. This is now implemented with built-in PyTorch linalg.lstsq function.
This function computes X = A.pinverse() @ B in a faster and more numerically stable way than performing the computations separately.
Computational efficiency
Computing gram matrices: We compute inner product matrics for code and math models on 10k training examples. Each of them take 3-hour on one Quadro RTX A6000 GPU (which can probably accelerated with more efficient LLM inference framework). But we have uploaded them under the merged model repo so that you do not need to re-compute.
Merging Models: ~2 minutes on the same GPU for this re-implementation. Please note loading two 8B models and (almost) equally sized inner product matrices at once can take >150GB memory.
Reproducing the results
Create a python environment and install the modified lm-eval-harness library for evaluating merged models.
cd lm-eval-harness
pip install -e .
The only modification is lm_eval/tasks/humaneval/humaneval.yaml.
Preparing activation inner product matrices.
You can download them from the merged model repo and place them under runs/merges/math-llama3/gram.pkl and runs/merges/code-llama3/gram.pkl. Alternatively, you can compute them yourself with,
python compute_gram.py code
python compute_gram.py math
Overall, simple averaging works well for LLMs and the benefits of merging algorithms diminishes for merging algorithms [1]
Citations
For the RegMean algorithm.
@inproceedings{
jin2023dataless,
title={Dataless Knowledge Fusion by Merging Weights of Language Models},
author={Xisen Jin and Xiang Ren and Daniel Preotiuc-Pietro and Pengxiang Cheng},
booktitle={The Eleventh International Conference on Learning Representations },
year={2023},
url={https://openreview.net/forum?id=FCnohuR6AnM}
}
Here are other useful references that greatly inspired this re-implementation.
