Views
No views yet
load_in_4bit=Truer=8 (LoRAランク)lora_alpha=16lora_dropout=0.05target_modules (LoRA適用モジュール)bias="none"task_type="CAUSAL_LM"modules_to_save=["embed_tokens"]1from peft import PeftModel
2from transformers import AutoModelForCausalLM, AutoTokenizer
3import torch
4
5# ベースモデルとトークナイザーのロード
6base_model_name = "tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.1"
7base_model = AutoModelForCausalLM.from_pretrained(base_model_name, torch_dtype=torch.bfloat16)
8tokenizer = AutoTokenizer.from_pretrained(base_model_name)
9
10# ファインチューニング済みモデルのロード
11model = PeftModel.from_pretrained(base_model, "yf591/Llama3SW8B_it_v0.1_qlora_FukuyamaBen")
12
13# テキスト生成
14prompt = "今日の天気は？"
15inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
16outputs = model.generate(**inputs, max_new_tokens=50)
17response = tokenizer.decode(outputs[0], skip_special_tokens=True)
18
19print(response)transformerspefttorchThe input hidden states seems to be silently casted in float32, this might be related to the fact you have upcasted embedding or layer norm layers in float32. We will cast back the input in torch.bfloat16.
[82/82 17:38, Epoch 0/1]
Step Training Loss
20 2.107600
40 1.558500
60 1.327600
80 1.270100
`rope_scaling`'s original_max_position_embeddings field must be less than max_position_embeddings, got 8192 and max_position_embeddings=8192
`rope_scaling`'s original_max_position_embeddings field must be less than max_position_embeddings, got 8192 and max_position_embeddings=8192
`rope_scaling`'s original_max_position_embeddings field must be less than max_position_embeddings, got 8192 and max_position_embeddings=8192
`rope_scaling`s original_max_position_embeddings field must be less than max_position_embeddings, got 8192 and max_position_embeddings=8192
`rope_scaling`'s original_max_position_embeddings field must be less than max_position_embeddings, got 8192 and max_position_embeddings=8192
`rope_scaling`'s original_max_position_embeddings field must be less than max_position_embeddings, got 8192 and max_position_embeddings=8192
CPU times: user 15min, sys: 2min 27s, total: 17min 28s
Wall time: 18min 2syf591/Llama3SW8B_it_v0.1_qlora_FukuyamaBen