In this work, we propose an extended framework of FuseLLM to integrate the collective knowledge and individual strengths of multiple structure and scale-varied chat LLMs into a more powerful chat LLM, resulting in FuseChat. FuseChat adopts a fuse-then-merge strategy with two main stages. Firstly, it undertakes pairwise knowledge fusion for source LLMs to derive multiple target LLMs of identical structure and size via lightweight fine-tuning. Then, these target LLMs are merged within the parameter space, wherein we propose a novel method VaRM for determining the merging weights based on the variation ratio of parameter matrices before and after fine-tuning.
Moreover, we argue that the concept of knowledge fusion adopted by both FuseChat and FuseLLM shares a fundamentally similar purpose with other related topics, such as the recently popular topic of mixture of experts (MoEs), because they all aim to leverage the strengths of multiple models (experts). However, while MoEs require loading multiple experts during inference, which has higher memory requirements, knowledge fusion supports the integration of multiple LLMs with diverse architectures into a single LLM without any additional memory requirement, making it more memory-efficient.
To support a plug-and-play fusion of new source LLM, we release our target LLMs: OpenChat-3.5-7B-Solar and OpenChat-3.5-7B-Mixtral, which are obtained from pair-wise knowledge fusion. Integrating a new source LLM at any scale requires only obtaining a target LLM from the new source LLM and merging it with the existing target LLMs.
We also release FuseChat with other merging methods: FuseChat-7B-SLERP and FuseChat-7B-TA, which achieves an average performance of 8.19 and 8.20 on MT-Bench respectively.
Here are the evaluation results.
Quick Start
Setup
We use python 3.11 in this project.
Then, we have to install all the libraries listed in requirements.txt.
pip install -r requirements.txt
Usage
Here's how you can run the model using the 🤗 Transformers:
The GPT4 template is also available as the integrated tokenizer.chat_template, which can be used instead of manually specifying the template:
python
1messages =[2{"role":"user","content":"Hello"},3{"role":"assistant","content":"Hi"},4{"role":"user","content":"How are you today?"}5]6tokens = tokenizer.apply_chat_template(messages, add_generation_prompt=True)7assert tokens ==[1,420,6316,28781,3198,3123,1247,28747,22557,32000,420,6316,28781,3198,3123,21631,28747,15359,32000,420,6316,28781,3198,3123,1247,28747,1602,460,368,3154,28804,32000,420,6316,28781,3198,3123,21631,28747]
Data Construction
We curated a comprehensive training dataset, FuseChat-Mixture, from various sources. This dataset covers different styles and capabilities, featuring both human-written and model-generated, and spanning general instruction-following and specific skills.
Here we show the scripts to obtain representations from multiple source LLMs for model fusion.
We evaluate FuseChat on MT-Bench, which comprises 80 multi-turn dialogues spanning writing, roleplay, reasoning, math, coding, stem, and humanities domains. Please download the official code and follow the guidelines for evaluation. We provide the scripts for our evaluation.
bash
1# Step 1. Generate model answers to MT-bench questions2exportCUDA_VISIBLE_DEVICES=0,13python gen_model_answer.py \4 --model-path "FuseAI/FuseChat-7B-VaRM"\5 --model-id "openchat_3.5_fusechat_7b_varm"\6 --num-gpus-per-model 1\7 --num-gpus-total 289# Step 2. Generate GPT-4 judgments10exportOPENAI_API_KEY=XXXXXX # set the OpenAI API key11python gen_judgment.py \12 --parallel 21314# Step 3. Show MT-bench scores15python show_result.py
Citation
If you find this work is relevant with your research or applications, please feel free to cite our work!
@article{wan2024fusechat,
title={FuseChat: Knowledge Fusion of Chat Models},
author={Fanqi Wan and Ziyi Yang and Longguang Zhong and Xiaojun Quan and Xinting Huang and Wei Bi},
journal={arXiv preprint arXiv:2402.16107},
year={2024}
}
