This model is a Fine-Tuned version of Qwen2.5-Coder-1.5B-Instruct using LoRA (Low-Rank Adaptation). It was trained as part of a Competitive Code Reasoning project.

🎯 Model Description

Model Type: LoRA Adapter
Base Model: Qwen/Qwen2.5-Coder-1.5B-Instruct
Focus: This specific adapter was trained on the DIVERSE dataset (containing a wide variety of problem types) to enhance general code problem-solving capabilities.

📊 Dataset

The model was fine-tuned on the CodeGen-Diverse-5K dataset.

Training Field: Sadece solution (code-only) alanı kullanılarak eğitilmiştir.
System Prompt: Training sırasında şu prompt zorunlu tutulmuştur:

"You are an expert Python programmer. Please read the problem carefully before writing any Python code."

⚙️ Hyperparameters

The following hyperparameters were used during training (as per project specifications):

Parameter	Value
Learning Rate	2e-4
Batch Size	8 (Effective: 16 via Gradient Accumulation)
Context Length	1024
LoRA Rank (r)	64
LoRA Alpha	128
LoRA Dropout	0.05
Target Modules	All Linear Layers (q, k, v, o, gate, up, down)
Optimizer	AdamW
Precision	bf16 (BFloat16)

📉 Training Results

Training loss logs demonstrating the model's convergence:

💻 How to Use

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# 1. Load Base Model
base_model_id = "Qwen/Qwen2.5-Coder-1.5B-Instruct"
model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# 2. Load This LoRA Adapter
adapter_id = "colak46/Qwen2.5-Coder-1.5B-DIVERSE-LoRA"
model = PeftModel.from_pretrained(model, adapter_id)
tokenizer = AutoTokenizer.from_pretrained(base_model_id)

# 3. Inference
prompt = "Write a Python function to solve a string manipulation problem."
messages = [
    {"role": "system", "content": "You are an expert Python programmer. Please read the problem carefully before writing any Python code."},
    {"role": "user", "content": prompt}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to("cuda")

outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))