Views
No views yet
1import torch
2from transformers import AutoModelForCausalLM, AutoTokenizer
3
4repo = "bunyaminergen/Qwen2.5-Coder-1.5B-Instruct-SFT-Distilled"
5tokenize = AutoTokenizer.from_pretrained(repo, padding_side="left")
6model = AutoModelForCausalLM.from_pretrained(
7 repo,
8 device_map="auto",
9 torch_dtype="auto",
10 ).eval()
11
12system = "You are a senior Python developer."
13user = "Give me a Python implementation of bubble sort."
14
15text = f"System: {system}\nUser: {user}\nAssistant:"
16inputs = tokenize(text, return_tensors="pt").to(model.device)
17
18with torch.no_grad():
19 out_ids = model.generate(**inputs, max_new_tokens=512)
20print(tokenize.decode(out_ids[0], skip_special_tokens=True))| Hyperparameter | Value |
|---|---|
| Base Model | bunyaminergen/Qwen2.5-Coder-1.5B-Instruct-SFT |
| Knowledge Distillation Method | Token based |
| Task Type | CAUSAL_LM |
| Number of Epochs | 11 |
| Batch Size | 12 |
| Gradient Accumulation Steps | 2 |
| Effective Batch Size | 24 (12 × 2) |
| Learning Rate | 5e-5 |
| Optimizer | AdamW |
| Precision | BF16 Mixed Precision |
| Evaluation Strategy | epoch |
| Max Sequence Length | 256 tokens |
| Logging Steps | every epoch steps |
| Save Checkpoint Steps | every 10000 steps |
| Experiment Tracking | MLflow (local) |
| Experiment Name | StudentKnowledgeDistillation |
| MLflow Run Name | StudentKD |
| Parameter | Value |
|---|---|
| Distillation Weight | 0.3 |
| Temperature | 0.5 |
| Loss Reduction | batchmean |
90%/10%42TrueTruemax_length=256)False10000 stepscheckpoint-10000checkpoint-13200 (final checkpoint)1@software{ Qwen2.5-Coder-1.5B-Instruct-SFT-Distilled,
2 author = {Bunyamin Ergen},
3 title = {{Qwen2.5-Coder-1.5B-Instruct-SFT-Distilled}},
4 year = {2025},
5 month = {04},
6}