This model is a fine-tuned version of the Qwen2.5-3B-Instruct architecture, optimized for instruction-following tasks. The fine-tuning process was conducted using the GRPO (Gradient Reversal Pretraining Optimization) method to enhance performance on specific queries.

Training Details

The fine-tuning was performed using the Unsloth Qwen2.5 (3B) GRPO Notebook. Key aspects of the training include:

Dataset: Custom dataset focusing on instruction-based queries.
Training Duration: Approximately 2 hours on a single GPU (T4 GPU - colab)
Challenges: Managed limited disk space by deleting intermediate files during the process.
Observation During Training: Reward increased as model answered questions correctly and completion_length became shorter.

Step - Training Loss - reward - reward_std - completion_length - kl

1 - 0.000000 - 0.125000 - 0.000000 - 200.000000 - 0.000000

2 - 0.000000 - 0.072375 - 0.248112 - 200.000000 - 0.000000

3 - 0.000000 - -0.079000 - 0.163776 - 182.500000 - 0.000005

Usage Instructions

To utilize this model for inference:

python

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("tradexy/ml-hamz-colab1-grpo_unsloth")
model = AutoModelForCausalLM.from_pretrained("tradexy/ml-hamz-colab1-grpo_unsloth")

input_text = "Your instruction here"
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))


For local deployment on Windows (can also use wsl ubuntu) see Machine Learning With Hamza
https://github.com/Hmzbo/Fine-tune-LLMS-with-grp