[!CAUTION] For optimal performance, we refrain from fine-tuning the model's identity. Thus, inquiries such as "Who are you" or "Who developed you" may yield random responses that are not necessarily accurate.

Updates

🚀🚀🚀 [July 24, 2024] We now introduce shenzhi-wang/Llama3.1-8B-Chinese-Chat! Compared to the original Meta-Llama-3.1-8B-Instruct model, our llama3.1-8B-Chinese-Chat model significantly reduces the issues of "Chinese questions with English answers" and the mixing of Chinese and English in responses. The training dataset contains >100K preference pairs, and it exhibits significant enhancements, especially in roleplay, function calling, and math capabilities!
🔥 We provide the official q4_0, q8_0, and f16 GGUF versions of Llama3.1-8B-Chinese-Chat-v2.1 at https://huggingface.co/shenzhi-wang/Llama3.1-8B-Chinese-Chat/tree/main/gguf!

Model Summary

llama3.1-8B-Chinese-Chat is an instruction-tuned language model for Chinese & English users with various abilities such as roleplaying & tool-using built upon the Meta-Llama-3.1-8B-Instruct model.

Developers: Shenzhi Wang*, Yaowei Zheng*, Guoyin Wang (in.ai), Shiji Song, Gao Huang. (*: Equal Contribution)

License: Llama-3.1 License
Base Model: Meta-Llama-3.1-8B-Instruct
Model Size: 8.03B
Context length: 128K (reported by Meta-Llama-3.1-8B-Instruct model, untested for our Chinese model)

1. Introduction

This is the first model specifically fine-tuned for Chinese & English users based on the Meta-Llama-3.1-8B-Instruct model. The fine-tuning algorithm used is ORPO [1].

Compared to the original Meta-Llama-3.1-8B-Instruct model, our llama3.1-8B-Chinese-Chat model significantly reduces the issues of "Chinese questions with English answers" and the mixing of Chinese and English in responses.

[1] Hong, Jiwoo, Noah Lee, and James Thorne. "Reference-free Monolithic Preference Optimization with Odds Ratio." arXiv preprint arXiv:2403.07691 (2024).

Training framework: LLaMA-Factory.

Training details:

epochs: 3
learning rate: 3e-6
learning rate scheduler type: cosine
Warmup ratio: 0.1
cutoff len (i.e. context length): 8192
orpo beta (i.e. $\lambda$ in the ORPO paper): 0.05
global batch size: 128
fine-tuning type: full parameters
optimizer: paged_adamw_32bit

2. Usage

2.1 Usage of Our BF16 Model

Please upgrade the transformers package to ensure it supports Llama3.1 models. The current version we are using is 4.43.0.
Use the following Python script to download our BF16 model

python

from huggingface_hub import snapshot_download
snapshot_download(repo_id="shenzhi-wang/Llama3.1-8B-Chinese-Chat", ignore_patterns=["*.gguf"])  # Download our BF16 model without downloading GGUF models.

Inference with the BF16 model