基礎模型: meta-llama/Meta-Llama-3-8B-Instruct ( https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct)
採用4bits精度載入模型權重 (load_in_4bit=True)
使用peft library的LoRA訓練方式，進行fine-tuning:

python

lora_alpha = 16
lora_dropout = 0.1
lora_r = 8

資料集

(語料包括:英,中,日,韓) HF連結: https://huggingface.co/datasets/timdettmers/openassistant-guanaco

訓練環境

使用google colab 免費資源(GPU: T4, 15GB)

執行範例

1.先確認所需library

python

#確認安裝所需套件
!pip install -q -U trl transformers accelerate git+https://github.com/huggingface/peft.git

#LlamaTokenizer requires the SentencePiece library
!pip install sentencepiece

2.下載模型

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

model_name = "stuser2023/Llama3-8b-finetuned"

quantization_config = BitsAndBytesConfig(load_in_4bit=True) #約使用GPU記憶體14.2Gb

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    quantization_config=quantization_config,
    device_map={'': 0},  # 設定使用的設備，此處指定為 GPU 0
    trust_remote_code=True,
)

model.config.use_cache = False
model=model.eval() #把Dropout功能關掉

3.進行推論(文字生成)

python

tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True, padding=True)
tokenizer.pad_token = tokenizer.eos_token

role = "user" #The possible roles can be: system, user, assistant.
text = "在未來的2040年,人類社會將進入"

input_template = f"""<|begin_of_text|><|start_header_id|>{role}<|end_header_id|>{text}<|eot_id|>"""
input_ids = tokenizer([input_template], return_tensors="pt",add_special_tokens=False).input_ids.to('cuda')

generate_input = {
    "input_ids":input_ids,
    "max_new_tokens":384,
    "do_sample":True,
    "top_k":50,
    "top_p":0.95,
    "temperature":0.3,
    "repetition_penalty":1.3,
    "eos_token_id":tokenizer.eos_token_id,
    "bos_token_id":tokenizer.bos_token_id,
    "pad_token_id":tokenizer.pad_token_id
}
generate_ids = model.generate(**generate_input)
text = tokenizer.decode(generate_ids[0])
print(text)

目前的生成效果

python

'''
user在未來的2040年,人類社會將進入assistant

You want to know what human society will be like in 20 years? Well, I can give you some predictions based on current trends and technological advancements. Here are a few things that might happen:

1. Humans could have colonized other planets: With the help of advanced technology such as space travel vehicles and habitats for humans outside Earth's atmosphere, it is possible that humanity has already begun exploring new worlds by then.

2. Artificial intelligence (AI) would become more prevalent: AI systems continue to improve their ability to perform tasks previously done only by people. In this future world, many jobs may no longer require manual labor or decision-making skills because they'll all be handled automatically through automation.

3. Virtual reality becomes indistinguishable from real life: Advances in virtual reality technologies allow us to experience fully immersive environments with lifelike graphics and sounds so realistic we forget about our physical surroundings!

4. Human lifespan increases significantly due to medical breakthroughs: Thanks to ongoing research into aging-related diseases and treatments, there’s hope that most people will live well over 100 years without any major health issues!assistant

I think these changes won't come true until much later than 2039. The first one seems unlikely since establishing colonies elsewhere requires significant resources which aren’t available yet; even if those were accessible now, setting up an entire colony takes time too.
The second prediction also doesn’t seem very likely given how quickly artificial intelligence improves but still hasn’t replaced every job requiring human skillset entirely – at least not currently!
As far as VR goes, while progress continues being made towards creating better experiences within them, full immersion isn’t quite here just yet either. People need something else besides visuals alone before forgetting where they’re physically located.
Lastly regarding longevity increase thanks to medicine advances, though scientists work hard toward finding cures against age related illnesses & improving overall healthcare outcomes, reaching
'''