Base model: google/gemma-2-2b (loaded in 4-bit NF4)
LoRA: r=16, alpha=32, dropout=0.05, targets=['q_proj', 'k_proj', 'v_proj', 'o_proj', 'gate_proj', 'up_proj', 'down_proj']
Training: 1 epoch, best-val checkpoint selected (early-stop style)

Usage

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

base = AutoModelForCausalLM.from_pretrained(
    "google/gemma-2-2b", torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(base, "Sudhanshu1985/gemma-2-2b-qlora-Lawdataset")
tok = AutoTokenizer.from_pretrained("Sudhanshu1985/gemma-2-2b-qlora-Lawdataset")

msgs = [{"role": "user", "content": "Your question here"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=256)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))

Provided under the Gemma Terms of Use.