Base model: Qwen/Qwen2.5-0.5B-Instruct
Method: full fine-tune (no LoRA), bf16 mixed precision
Data: GSM8k train, chat format (system + user); assistant target is the full worked solution ending in #### N
Epochs: 3 (348 steps); AdamW, lr 1e-5 cosine, warmup ratio 0.1, weight decay 0.1, grad clip 1.0
Max sequence length: 1024

Prompt format

System prompt used during training; the model ends its answer with #### <number>:

You are a math problem solver. Think step by step. You MUST end your response with '#### ' where is the final numerical answer (digits only, no units or markdown).

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

m = AutoModelForCausalLM.from_pretrained("OhhMoo/qwen05b-gsm8k-sft-instruct")
tok = AutoTokenizer.from_pretrained("OhhMoo/qwen05b-gsm8k-sft-instruct")

messages = [
    {"role": "system", "content": "You are a math problem solver. Think step by step. You MUST end your response with '#### <number>' where <number> is the final numerical answer (digits only, no units or markdown)."},
    {"role": "user", "content": "Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? Let's think step by step."},
]
ids = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
out = m.generate(ids, max_new_tokens=512, do_sample=False)
print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))