You can deploy this model and start earning money today!

ewre324/
ewre324-QwQ-0.5B-Distilled-SFT-Reason

Name: ewre324-QwQ-0.5B-Distilled-SFT-Reason
Availability: InStock

safetensors

qwen2

KingNish/reasoning-base-20k

apache-2.0

Views

No views yet

Fine-tuned kz919/QwQ-0.5B-Distilled-SFT upon KingNish/reasoning-base-20k

Original Model Card Follows:

license: apache-2.0 datasets:

amphora/QwQ-LongCoT-130K-2
PowerInfer/QWQ-LONGCOT-500K
PowerInfer/LONGCOT-Refine-500K language:
en metrics:
perplexity base_model:
Qwen/Qwen2.5-0.5B-Instruct library_name: transformers

Model Details:

Base Model: Qwen/Qwen2-0.5B-Instruct
Teacher Model: Qwen/QwQ-32B-Preview
Distillation Framework: Instruction Tuning
Task Type: Conversational AI / Causal Language Modeling
Parameters: 0.5B
Special Features:
- Integrated gradient checkpointing for efficient training
- Step-by-step reasoning capabilities for better problem-solving

Training:

QwQ-0.5B-Distilled was trained using the QwQ-LongCoT-130K dataset, a carefully curated collection of long-context examples designed for reasoning and conversational AI tasks. The GKD framework ensures that the student model mimics the teacher model’s outputs, aligning its predictions with high-quality responses.

Training Progress:

[▓▓▓▓▓▓▓▓▓▓] 100%

Training Script:

python

import os
import argparse
import torch
from datasets import Dataset
from trl import SFTConfig, SFTTrainer, DataCollatorForCompletionOnlyLM
from transformers import (
    AutoModelForCausalLM,
    AutoTokenizer,
)
from datasets import load_dataset
from peft import LoraConfig

parser = argparse.ArgumentParser()
parser.add_argument("--max_length", type=int, default = 4096)
parser.add_argument("--output_dir", type=str, default="gkd-model")
parser.add_argument("--per_device_train_batch_size", type=int, default=1)
parser.add_argument("--gradient_accumulation_steps", type=int, default=16)
parser.add_argument("--gradient_checkpointing", action="store_true", default=False)
parser.add_argument("--resume_from_checkpoint", action="store_true", default=False)
parser.add_argument("--lora", action="store_true")
args = parser.parse_args()

qwq_dataset = load_dataset("amphora/QwQ-LongCoT-130K-2", split = "train")
messages = []
for each in qwq_dataset:
    msg = [
        {"role": "system", "content": "You are a helpful and harmless assistant. You are Qwen developed by Alibaba. You should think step-by-step."},
        {"role": "user", "content": each["problem"]},
        {"role": "assistant", "content": each["qwq"]},
    ]
    messages.append(msg)

TRAIN_SPLIT_RATIO = 0.9
train_size = int(TRAIN_SPLIT_RATIO * len(messages))
eval_size = len(messages) - train_size

tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct")

# The model to optimise
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct", torch_dtype=torch.bfloat16, device_map="auto") 



### Real Dataset
train_dataset = Dataset.from_dict({"messages":messages[:train_size]})
eval_dataset = Dataset.from_dict({"messages":messages[train_size:]})
training_args = SFTConfig(
    output_dir=args.output_dir,
    max_seq_length=args.max_length,
    per_device_train_batch_size=args.per_device_train_batch_size,
    gradient_accumulation_steps=args.gradient_accumulation_steps,
    gradient_checkpointing = args.gradient_checkpointing,
    save_steps = 100,
    save_total_limit = 5
    )

lora_config = LoraConfig(
    r=16,
    lora_alpha=32,
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM",
)

response_template = "<|im_start|>assistant\n"

collator = DataCollatorForCompletionOnlyLM(response_template, tokenizer=tokenizer)

trainer = SFTTrainer(
    model=model,
    args=training_args,
    processing_class=tokenizer,
    train_dataset=train_dataset,
    eval_dataset=eval_dataset,
    peft_config=lora_config if args.lora else None,
    data_collator=collator,
)
trainer.train(resume_from_checkpoint=args.resume_from_checkpoint)

Dataset:

Source: amphora/QwQ-LongCoT-130K
Split: 90% Training, 10% Evaluation

Example Usage: