Parameter	Value
Base model	Qwen3-8B (4-bit quantized via Unsloth)
Fine-tuning method	LoRA (Low-Rank Adaptation)
LoRA rank (r)	64
LoRA alpha	128
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Training epochs	3
Total steps	915
Batch size	2
Final training loss	0.288
Eval loss (epoch 1)	0.840
Eval loss (epoch 2)	0.755
Eval loss (epoch 3)	0.804

Usage

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = AutoModelForCausalLM.from_pretrained(
    "unsloth/qwen3-8b-bnb-4bit",
    device_map="auto",
)
model = PeftModel.from_pretrained(base_model, "KenWuqianghao/LeLM")
tokenizer = AutoTokenizer.from_pretrained("KenWuqianghao/LeLM")

messages = [
    {"role": "user", "content": "Fact check this NBA take: LeBron is washed"}
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
outputs = model.generate(inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training

Trained with TRL SFT (Supervised Fine-Tuning) using Unsloth for efficient LoRA training.

Framework Versions

PEFT: 0.18.1
TRL: 0.24.0
Transformers: 4.57.6
PyTorch: 2.10.0+cu128
Datasets: 4.3.0
Tokenizers: 0.22.2

Part of LeGM-Lab

This model powers LeGM-Lab, an LLM-powered NBA take analysis and roasting bot.

KenWu/LeLM

LeLM - NBA Take Analysis Language Model

Model Details

Usage

Training

Framework Versions

Part of LeGM-Lab

KenWu/
LeLM