Views
No views yet
| Parameter | Value |
|---|---|
| Base Model | meta-llama/Llama-3.2-3B |
| Fine-tuning Method | LoRA |
| Trainer | TRL SFTTrainer |
| Epochs | 1 |
| Learning Rate | 2e-4 |
| Per Device Train Batch Size | 1 |
| Gradient Accumulation Steps | 8 |
| Effective Batch Size | 8 |
| Max Sequence Length | 512 |
| Optimizer | paged_adamw_8bit |
| Warmup Steps | 100 |
| Gradient Checkpointing | Enabled |
| Packing | Enabled |
| Mixed Precision | BF16 |
1from transformers import AutoModelForCausalLM, AutoTokenizer
2from peft import PeftModel
3
4BASE_MODEL = "meta-llama/Llama-3.2-3B"
5
6tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)
7
8base_model = AutoModelForCausalLM.from_pretrained(BASE_MODEL)
9
10model = PeftModel.from_pretrained(
11 base_model,
12 "ruhil6789/solana-llama-3.2-3b-lora"
13)
14
15prompt = "Explain Program Derived Addresses (PDAs) in Solana."
16
17inputs = tokenizer(prompt, return_tensors="pt")
18
19outputs = model.generate(**inputs, max_new_tokens=200)
20
21print(tokenizer.decode(outputs[0], skip_special_tokens=True))
22
23print(tokenizer.decode(outputs[0], skip_special_tokens=True))