LoRA supervised fine-tuning of Qwen/Qwen3-0.6B on nvidia/OpenMathReasoning, merged into a standalone full model (16-bit). Part of an IBA NLP Assignment 04 LoRA ablation study (trials T1-T5).

Trial T2: Data efficiency -- 50% of 500 (250 rows)

Hyperparameter	Value
LoRA rank	16
Data fraction	50% of 500 rows
Learning rate	0.0002
Epochs	1
Target modules	q_proj, k_proj, v_proj, o_proj
Max seq length	2048

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("COMPUTATION-NOT-FOUND/qwen3-0.6b-datasetB-T2")
tok   = AutoTokenizer.from_pretrained("COMPUTATION-NOT-FOUND/qwen3-0.6b-datasetB-T2")
msgs  = [{"role": "user", "content": "What is 2+2?"}]
ids   = tok.apply_chat_template(msgs, return_tensors="pt",
                                add_generation_prompt=True, enable_thinking=True)
print(tok.decode(model.generate(ids, max_new_tokens=512)[0]))