OpenChat: Advancing Open-source Language Models with Imperfect Data
The OpenChat v2 family is inspired by offline reinforcement learning, including conditional behavior cloning (OpenChat-v2) and weighted behavior cloning (OpenChat-v2-w).
- OpenChat-v2-w: ~80k cleaned ShareGPT data with conditioning and weighted loss, based on LLaMA-13B with a context length of 2048.
- Achieves 50.9% win-rate over ChatGPT on MT-bench.
- Achieves 79.4% win-rate over ChatGPT on Vicuna-bench.
- Achieves 87.1% win-rate over text-davinci-003 on AlpacaEval.
- OpenChat-v2: ~80k cleaned ShareGPT data with only conditioning, based on LLaMA-13B with a context length of 2048.
- Achieves 48.1% win-rate over ChatGPT on MT-bench.
- Achieves 80.6% win-rate over ChatGPT on Vicuna-bench.
- Achieves 85.0% win-rate over text-davinci-003 on AlpacaEval.
Code and Inference Server
We provide the full source code, including an inference server compatible with the "ChatCompletions" API, in the
OpenChat GitHub repository.
Web UI
OpenChat also includes a web UI for a better user experience. See the GitHub repository for instructions.
Conversation Template
The conversation template involves concatenating tokens, and cannot be expressed in plain-text.
Besides base model vocabulary, an end-of-turn token <|end_of_turn|> is added.
Here is an example of single-round conversation template:
1def tokenize_single_input(tokenizer, prompt):
2 # OpenChat V2
3 human_prefix = "User:"
4 prefix = "Assistant GPT4:"
5 eot_token = "<|end_of_turn|>"
6 bos_token = "<s>"
7
8 def _tokenize(text):
9 return tokenizer.convert_tokens_to_ids(tokenizer._tokenize(text))
10
11 def _tokenize_special(special_name):
12 return tokenizer.convert_tokens_to_ids(special_name)
13
14 return [_tokenize_special(bos_token)] + _tokenize(human_prefix) + _tokenize(prompt) + [_tokenize_special(eot_token)] + \
15 _tokenize(prefix)
To explore conditional language models, you can also set prefix = "Assistant GPT3:" to mimic ChatGPT behavior (this may cause performance degradation).
Hint: In BPE, tokenize(A) + tokenize(B) does not always equals to tokenize(A + B)
Limitations
Foundation Model Limitations
Despite its advanced capabilities, OpenChat is still bound by the limitations inherent in its foundation models. These limitations may impact the model's performance in areas such as:
- Complex reasoning
- Mathematical and arithmetic tasks
- Programming and coding challenges
Hallucination of Non-existent Information
OpenChat may sometimes generate information that does not exist or is not accurate, also known as "hallucination". Users should be aware of this possibility and verify any critical information obtained from the model.
