Views
No views yet
TinyLlama (base)
│
├── Stage 1: Domain Pretraining (LoRA)
│ [ThakrePranjal/pharma-tinyllama-domain-lora]
│
├── Stage 1 Merged Model → used as base for Stage 2
│
├── Stage 2: Instruction Fine-Tuning (LoRA / SFT)
│ [ThakrePranjal/pharma-tinyllama-instruct-lora]
│
├── Stage 2 Merged Model → used as base for Stage 3
│ [ThakrePranjal/pharma-tinyllama-instruct-merged]
│
└── Stage 3: DPO Preference Tuning (LoRA)
[THIS ADAPTER]1from transformers import AutoModelForCausalLM, AutoTokenizer
2from peft import PeftModel
3import torch
4
5# Load Stage 2 merged model as base
6base_model = AutoModelForCausalLM.from_pretrained(
7 "ThakrePranjal/pharma-tinyllama-instruct-merged"
8)
9
10# Attach Stage 3 DPO LoRA adapter
11model = PeftModel.from_pretrained(base_model, "ThakrePranjal/pharma-tinyllama-dpo-lora")
12tokenizer = AutoTokenizer.from_pretrained("ThakrePranjal/pharma-tinyllama-dpo-lora")
13model.eval()1def generate(instruction, input_text="", max_new_tokens=150):
2 if input_text.strip():
3 prompt = (
4 f"### Instruction:\n{instruction}\n\n"
5 f"### Input:\n{input_text}\n\n"
6 f"### Response:\n"
7 )
8 else:
9 prompt = f"### Instruction:\n{instruction}\n\n### Response:\n"
10
11 inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
12 with torch.no_grad():
13 out = model.generate(
14 **inputs,
15 max_new_tokens=max_new_tokens,
16 do_sample=True,
17 temperature=0.7,
18 top_p=0.9,
19 repetition_penalty=1.1,
20 pad_token_id=tokenizer.eos_token_id,
21 )
22 return tokenizer.decode(out[0], skip_special_tokens=True)
23
24print(generate("Explain the primary mechanism of action of metformin."))| Param | Value |
|---|---|
| Beta (KL penalty) | 0.1 |
| Epochs | 3 (max_steps=5) |
| Learning rate | 5e-5 |
| Batch size | 1 |
| Grad accum steps | 8 |
| Max seq length | 512 |
| Max prompt length | 256 |
| LoRA rank (r) | 16 |
| LoRA alpha | 32 |