This repository contains the fine-tuned weights for Qwen/Qwen2.5-7B-Instruct trained to produce explicit step-by-step <think>...</think> Chain-of-Thought (CoT) reasoning.

Model Summary

Fine-Tuning Methodology: Method 1: QLoRA (4-bit NF4 Quantization + 16-bit LoRA Adapters)
Base Model: Qwen/Qwen2.5-7B-Instruct
Trainable Parameters: 40.37 Million (0.53% trainable parameters)
Dataset: FreedomIntelligence/medical-o1-reasoning-SFT (10,000 instructions)
Hardware Acceleration: 2x NVIDIA H200 NVL GPUs (288 GB VRAM)

Empirical Benchmark Performance

Metric / Feature	Benchmark Score
CoT Reasoning Tag Rate (`<think>`)	`25.0%`
Holdout Benchmark Accuracy	`100.0%`
Training Duration	`19.77 minutes`
Final Epoch Loss	`1.520`

Usage Instructions

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "Qwen/Qwen2.5-7B-Instruct"
adapter_model_id = "DexterSptizu/qwen2.5-7b-reasoning-qlora"

# Load base model & tokenizer
tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Attach fine-tuned reasoning adapter
model = PeftModel.from_pretrained(base_model, adapter_model_id)

# Inference Example
prompt = "<|im_start|>system\nYou are a helpful reasoning assistant.<|im_end|>\n<|im_start|>user\nSolve step by step: What is 15 * 24?<|im_end|>\n<|im_start|>assistant\n<think>\n"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Citation & Acknowledgements

Base model developed by Alibaba Cloud (Qwen Team).
Fine-tuning executed on dual NVIDIA H200 NVL GPUs.

namanadep/qwen2.5-7b-instruct-reasoning-qlora

Qwen2.5-7B Reasoning (QLoRA 4-bit Adapter)

Model Summary

Empirical Benchmark Performance

Usage Instructions

Citation & Acknowledgements

namanadep/
qwen2.5-7b-instruct-reasoning-qlora