Views
No views yet
<|im_start|>user
[文章內容] ← ❌ 不計算 loss
<|im_end|>
<|im_start|>assistant
<think>
[推理過程] ← ✅ 計算 loss
</think>
答案: X ← ✅ 計算 loss
<|im_end|>| 參數 | 值 |
|---|---|
| Base Model | unsloth/DeepSeek-R1-Distill-Llama-70B-bnb-4bit |
| LoRA Rank | 256 |
| LoRA Alpha | 256 |
| Learning Rate | 2e-05 |
| Effective Batch Size | 32 |
| Epochs | 1 |
| Max Sequence Length | 4096 |
1from unsloth import FastLanguageModel
2
3model, tokenizer = FastLanguageModel.from_pretrained(
4 "kunhsiang/RC-Competition-DeepSeek-R1-Distill-Llama-70B-bnb-4bit-SFTOnly-ChatML-CoT-v2-20251211_0619",
5 max_seq_length=4096,
6 load_in_4bit=True,
7)
8FastLanguageModel.for_inference(model)
9
10messages = [{"role": "user", "content": "[文章內容]\n問題: ...\n選項: 1. ... 2. ... 3. ... 4. ..."}]
11inputs = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt").to("cuda")
12outputs = model.generate(inputs, max_new_tokens=1024, temperature=0.7, top_p=0.9)
13print(tokenizer.decode(outputs[0], skip_special_tokens=False))