You can deploy this model and start earning money today!

dongwookkwon/
qwen0.5b-tech-interview-test

Name: qwen0.5b-tech-interview-test
Availability: InStock

Views

No views yet

qwen0.5b-tech-interview-test

This model is a fine-tuned version of Qwen/Qwen2.5-0.5B on mathematical reasoning tasks. It has been trained using TRL with QLoRA (Quantized LoRA).

Model Details

Base Model: Qwen/Qwen2.5-0.5B
Fine-tuning Method: QLoRA (Quantized LoRA) followed by weight merging
Task: Mathematical reasoning (GSM8K benchmark)
Training Framework: TRL (Transformer Reinforcement Learning)

Training Data

The model was fine-tuned on a mixture of datasets:

GSM8K (15.7%): 7,473 samples from the GSM8K training set (human-written natural reasoning)
NuminaMath-CoT (84.3%): 40,000 samples from the NuminaMath-CoT dataset (model-generated CoT examples)

Total training samples: 47,473 Train/Test Split: 90%/10% (42,726 train / 4,747 test)

Dataset Composition Strategy

The combination strategy aimed to balance:

Natural human reasoning patterns from GSM8K
Diverse Chain-of-Thought (CoT) patterns from NuminaMath-CoT

Both datasets were converted to a unified messages format compatible with Qwen's chat template.

Evaluation Results

GSM8K Benchmark

Metric	Method	Few-shot	Score	Std Error
exact_match	flexible-extract	5	34.12%	±1.31%
exact_match	strict-match	5	33.59%	±1.30%

Baseline (Qwen2.5-0.5B-Instruct): 34.42% (flexible-extract), 31.69% (strict-match)
Improvement:
- Flexible-extract: Comparable performance (34.12% vs 34.42%)
- Strict-match: +1.90% improvement (33.59% vs 31.69%)
Note: This model was fine-tuned on a curated dataset mixture of 47,473 samples to improve mathematical reasoning capabilities

Evaluation Details

Evaluation Tool: EleutherAI's lm-evaluation-harness
Inference Engine: vLLM (for efficient batch inference)
Test Samples: 1,319 (GSM8K test split)
Generation Settings:
- temperature=0.0
- do_sample=False
- max_tokens=256
Evaluation Method: Few-shot evaluation with 5 examples
Data Leakage Prevention: Only GSM8K test split used for evaluation, train split was used for training

Training Procedure

Training Hyperparameters

Learning Rate: 2e-5 (increased from 5e-6 for faster convergence)
Training Epochs: 2 (with early stopping)
Batch Size: 1 (per device)
Effective Batch Size: 8 (with gradient accumulation)
Gradient Accumulation Steps: 8 (increased from 4 for stable gradients)
Weight Decay: 0.01
Max Gradient Norm: 1.0
Max Sequence Length: 2048
Warmup Ratio: 0.15 (increased from 0.05 for better training stability)

QLoRA Configuration

Quantization: 8-bit (BitsAndBytes)
Quantization Config:
- llm_int8_threshold=6.0
- llm_int8_has_fp16_weight=False
- llm_int8_enable_fp32_cpu_offload=False
LoRA Rank (r): 32 (increased from 16 for more capacity)
LoRA Alpha: 64 (increased from 32, typically 2x rank)
LoRA Dropout: 0.1
Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Trainable Parameters: ~17.6M (3.4% of total parameters: 511.6M)
Gradient Checkpointing: Enabled (for memory efficiency)

Training Process

The model was trained using:

Training Framework: TRL SFTTrainer with QLoRA
Data Formatting: Qwen chat template applied to messages format
Evaluation Strategy: Steps (every 250 steps)
Checkpoint Saving: Every 500 steps
Early Stopping: Enabled with patience=3 (based on eval_loss)
Best Model Selection: Based on lowest eval_loss
Optimizer: paged_adamw_8bit (8-bit AdamW optimizer for memory efficiency)
Learning Rate Schedule: Cosine decay
Packing: Enabled (for efficient batch processing)
Model Merging: LoRA weights merged with base model after training for inference

Key Optimizations

Dataset Curation: Combined GSM8K (human-written) and NuminaMath-CoT (model-generated) for balanced learning
Hyperparameter Tuning: Increased learning rate and warmup ratio for better convergence
Memory Efficiency: 8-bit quantization + gradient checkpointing + LoRA adapters
Training Stability: Gradient accumulation and early stopping to prevent overfitting

Model Usage

Using Transformers

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "dongwookkwon/qwen0.5b-tech-interview-test"
model = AutoModelForCausalLM.from_pretrained(model_name, trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)

# Format your question
question = "Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May?"

messages = [
    {"role": "user", "content": question}
]

# Apply chat template
inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt"
)

# Generate
outputs = model.generate(
    inputs,
    max_new_tokens=256,
    temperature=0.0,
    do_sample=False
)

response = tokenizer.decode(outputs[0], skip_special_tokens=False)
print(response)

Using vLLM (for faster inference)

python

from vllm import LLM, SamplingParams

model = LLM(
    model="dongwookkwon/qwen0.5b-tech-interview-test",
    trust_remote_code=True,
    dtype="float16",
    gpu_memory_utilization=0.5
)

sampling_params = SamplingParams(
    temperature=0.0,
    max_tokens=256
)

prompt = "Question: Natalia sold clips to 48 of her friends in April..."
outputs = model.generate([prompt], sampling_params)

Limitations

Domain Specificity: This model is fine-tuned specifically for mathematical reasoning tasks and may not perform well on other domains
Model Size: The 0.5B parameter size limits reasoning capabilities compared to larger models (7B+)
Problem Complexity: Performance may vary depending on the complexity of mathematical problems
Data Dependency: Model performance is dependent on the quality and diversity of the training data mixture
Inference Requirements: While optimized for inference, the model still requires GPU resources for best performance

Training Infrastructure

Framework Versions

TRL: 0.24.0 (SFTTrainer)
Transformers: 4.57.1
PyTorch: 2.8.0
Datasets: 4.3.0
PEFT: Latest (for LoRA/QLoRA support)
BitsAndBytes: Latest (for 8-bit quantization)
Accelerate: >=0.26.0
lm-evaluation-harness: 0.4.9.1 (for evaluation)
vLLM: Latest (for efficient batch inference during evaluation)

Hardware Requirements

Training: GPU with CUDA support (tested on A100, T4)
Inference: GPU recommended for best performance
Memory: ~8GB VRAM minimum for 8-bit QLoRA training

Citation

If you use this model, please cite:

bibtex

@misc{qwen0.5b-tech-interview-test,
  title={qwen0.5b-tech-interview-test: Fine-tuned Qwen2.5-0.5B for Mathematical Reasoning},
  author={Dongwook Kwon},
  year={2024},
  howpublished={\url{https://huggingface.co/dongwookkwon/qwen0.5b-tech-interview-test}}
}

Base Model Citation

bibtex

@misc{qwen2.5,
  title={Qwen2.5: A Party of Foundation Models},
  author={Qwen Team},
  year={2024},
  howpublished={\url{https://huggingface.co/Qwen/Qwen2.5-0.5B}}
}

Dataset Citations

GSM8K: Cobbe et al., 2021
NuminaMath-CoT: AI-MO/NuminaMath-CoT

Acknowledgments

This model was developed as part of a coding challenge focused on optimizing small language models for mathematical reasoning tasks. The approach combines efficient fine-tuning techniques (QLoRA) with curated dataset mixtures to improve performance on the GSM8K benchmark.