Temperature: 2.0 (optimal)
Alpha: 0.95 (95% distillation weight)
LoRA Rank: 8
Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj

Usage

python

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

# Load base model and tokenizer
base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-Coder-3B-Instruct-AWQ",
    torch_dtype=torch.float16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-Coder-3B-Instruct-AWQ")

# Load distilled adapter
model = PeftModel.from_pretrained(base_model, "Vinitha2004/qwen2.5-coder-3b-instruct-awq-final-working_draft")

# Generate code
input_text = "Original Code:\ndef add(a, b):\n    return a + b\n\nUpdate Snippet:\n// ... existing code ...\ndef add(a: int, b: int) -> int:\n// ... existing code ...\n\nUpdated Code:\n"
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100)
result = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(result)

Performance

This distilled model retains the knowledge from the 7B teacher model while being significantly more efficient:

Faster inference (3B vs 7B parameters)
Lower memory usage
Maintained code generation quality

Training Dataset

Trained on 5000 code editing examples from custom dataset.

Files

adapter_config.json: LoRA configuration
adapter_model.safetensors: Trained LoRA weights (59MB)
Other standard tokenizer files