The goal of this project is to create a small, efficient, domain-focused assistant for Python engineering, frontend development, debugging, refactoring, and production-minded code explanations.

This is a LoRA adapter, not a fully merged standalone model.

Project Motivation

Most small coding models can generate code, but they often struggle with structured debugging, practical explanations, and production-oriented guidance.

CodeMate was fine-tuned to respond like a senior engineering assistant by encouraging outputs such as:

Clear implementation plans
Self-contained code
Root-cause debugging
Practical explanations
Frontend-focused React/Next.js/Tailwind examples
Pythonic, readable, type-aware solutions

Base Model

Qwen/Qwen2.5-Coder-1.5B-Instruct

Training Method

The model was fine-tuned using:

Unsloth
QLoRA / 4-bit loading
PEFT LoRA adapters
SFTTrainer
Cosine learning rate schedule
AdamW 8-bit optimizer

LoRA configuration:

text

rank: 16
lora_alpha: 32
target_modules:
- q_proj
- k_proj
- v_proj
- o_proj
- gate_proj
- up_proj
- down_proj

Usage

Install dependencies:

pip install unsloth transformers peft accelerate bitsandbytes

Load the LoRA adapter:

python

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="micymike/codemate-qwen-lora",
    max_seq_length=2048,
    load_in_4bit=True,
)

Example Output

Prompt:

Write a Python function to safely merge two dictionaries by summing matching numeric values.

Example response:

python

def safe_merge(dict1: dict, dict2: dict) -> dict:
    """Safely merges two dictionaries by summing values of keys that exist in both."""
    merged_dict = {}

    for key in set(dict1.keys()).union(set(dict2.keys())):
        if key in dict1 and key in dict2:
            merged_dict[key] = dict1[key] + dict2[key]
        elif key in dict1:
            merged_dict[key] = dict1[key]
        else:
            merged_dict[key] = dict2[key]

    return merged_dict

Author

Built by Michael Moses / micymike as part of a practical AI engineering project focused on LLM fine-tuning, coding assistants, dataset curation, and deployment.