This is a LoRA (Low-Rank Adaptation) fine-tuned adapter for Qwen2.5-0.5B-Instruct, trained on a subset of the CodeAlpaca-20k dataset to improve Python code generation from natural language instructions.

Model Details

Base model: Qwen/Qwen2.5-0.5B-Instruct
Fine-tuning method: LoRA (via PEFT)
Dataset: CodeAlpaca-20k (subset of 50 examples)
Training hardware: CPU-only (no GPU)
LoRA config: r=8, alpha=16, target modules: q_proj, v_proj

Intended Use

This adapter is intended for generating short Python functions and code snippets from natural language instructions. It was built as a learning project to understand the fine-tuning pipeline (dataset prep, LoRA, training loop) rather than for production use.

How to Use

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-0.5B-Instruct", dtype=torch.float32
)
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct")
model = PeftModel.from_pretrained(base_model, "salik702/qwen2.5-0.5b-codealpaca-lora")

prompt = "### Instruction:\nWrite a Python function to check if a number is prime\n\n### Response:\n"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=150)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Limitations

Trained on only 200 examples for a small number of epochs — this is a learning/demo project, not a production-grade coding assistant.
May overfit to patterns seen in the small training subset rather than generalizing broadly.
Base model itself (0.5B parameters) has limited reasoning capacity compared to larger code models.

Training Details

Fine-tuned using Hugging Face transformers + peft on a CPU-only machine, using a custom training script with LoRA adapters applied to attention projection layers.

License

MIT

salikahmad702/qwen2.5-0.5b-codealpaca-lora

Qwen2.5-0.5B CodeAlpaca LoRA