Chinese-Alpaca-Plus-13B-GPTQ

Model Details

Model Description

• Developed by: ymcui (Yiming Cui)
• Shared by: Known Rabbit
• Language(s) (NLP): Chinese, English
• License: Apache 2.0
• Finetuned from model: LLaMA

In order to promote the open research of large models in the Chinese NLP community, this project open sourced the Chinese LLaMA model and the Alpaca large model with fine-tuned instructions. Based on the original LLaMA, these models expand the Chinese vocabulary and use Chinese data for secondary pre-training, which further improves the basic semantic understanding of Chinese. At the same time, the Chinese Alpaca model further uses Chinese instruction data for fine-tuning, which significantly improves the model's ability to understand and execute instructions. For details, please refer to the technical report (Cui, Yang, and Yao, 2023).

Model Sources

• Repository: https://github.com/ymcui/Chinese-LLaMA-Alpaca
• Paper: [2304.08177] Efficient and Effective Text Encoding for Chinese LLaMA and Alpaca

Uses

Direct Use

How to easily download and use this model in text-generation-webui

1. Click the Model tab.
2. Under Download custom model or LoRA, enter rabitt/Chinese-Alpaca-Plus-13B-GPTQ.
3. Click Download.
4. Wait until it says it's finished downloading.
5. Click the Refresh icon next to Model in the top left.
6. In the Model drop-down: choose the model you just downloaded, Chinese-Alpaca-Plus-13B-GPTQ.
7. If you see an error like Error no file named pytorch_model.bin ... in the bottom right, ignore it - it's temporary.
8. Fill out the GPTQ parameters on the right: Bits = 4, Groupsize = 128, model_type = Llama
9. Click Save settings for this model in the top right.
10. Click Reload the Model in the top right.
11. Once it says it's loaded, click the Text Generation tab and enter a prompt!

Training Details

Training Procedure

1. Download models from the following links
   • Original LLaMA: https://github.com/facebookresearch/llama/pull/73
   • Chinese-LLaMA-Plus-13B
     • ziqingyang/chinese-llama-plus-lora-13b · Hugging Face
     • chinese_llama_plus_lora_13b.zip_免费高速下载|百度网盘-分享无限制
   • Chinese-Alpaca-Plus-13B
     • ziqingyang/chinese-alpaca-plus-lora-13b · Hugging Face
     • chinese_alpaca_plus_lora_13b.zip_免费高速下载|百度网盘-分享无限制
2. Convert LLaMA to HuggingFace (HF) format with convert_llama_weights_to_hf.py
   bash

   1wget https://github.com/huggingface/transformers/raw/main/src/transformers/models/llama/convert_llama_weights_to_hf.py 2PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python \ 3python convert_llama_weights_to_hf.py \ 4 --input_dir ./llama \ 5 --model_size 13B \ 6 --output_dir ./llama-13b-hf
3. Merge Chinese-LLaMA-Plus-13B and Chinese-Alpaca-Plus-13B into LLaMA with merge_llama_with_chinese_lora.py
   bash

   1wget https://github.com/ymcui/Chinese-LLaMA-Alpaca/raw/main/scripts/merge_llama_with_chinese_lora.py 2python merge_llama_with_chinese_lora.py \ 3 --base_model ./llama-13b-hf \ 4 --lora_model ./Chinese-LLaMA-Plus-LoRA-13B,./Chinese-Alpaca-Plus-LoRA-13B \ 5 --output_type huggingface \ 6 --output_dir ./Chinese-Alpaca-Plus-13B
4. Quantise the model with GPTQ-for-LLaMa
   bash

   1mkdir -p Chinese-Alpaca-Plus-13B-GPTQ 2git clone https://github.com/qwopqwop200/GPTQ-for-LLaMa.git 3cd GPTQ-for-LLaMa 4# export CUDA_VISIBLE_DEVICES=0 5python llama.py ../Chinese-Alpaca-Plus-13B c4 --wbits 4 --true-sequential --act-order --groupsize 128 --save_safetensors ../Chinese-Alpaca-Plus-13B-GPTQ/Chinese-Alpaca-Plus-13B-GPTQ-4bit-128g.safetensors

Citation