You can deploy this model and start earning money today!

rswaminathan38/
llmbench-student-3b-gsm8k-topk-kd-ft-teacher-20260410

Name: llmbench-student-3b-gsm8k-topk-kd-ft-teacher-20260410
Availability: InStock

safetensors

llama

Views

No views yet

Student 3B Top-K KD

This repo contains the top-k KD export for the student model from the GSM8K workflow in this project.

Quick Use

Transformers:

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "rswaminathan38/llmbench-student-3b-gsm8k-topk-kd-ft-teacher-20260410"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto")

vLLM:

vllm serve rswaminathan38/llmbench-student-3b-gsm8k-topk-kd-ft-teacher-20260410 --dtype auto

Metrics

Test relaxed exact-match accuracy: 0.2055
Correct / examples: 271 / 1319
Avg generated tokens: 253.0675
Prompt style used during evaluation: cot_step_by_step

Training Details

Base model: meta-llama/Llama-3.2-3B
Variant: top-k KD
Output source: /storage/ice1/3/3/rswaminathan38/LLM_Bench/LLMOptimization/Model_Optimizations/outputs/hf_topk_kd_teacher_2026-04-10
Teacher used for distillation: Fine-tuned GSM8K teacher 8B (2026-04-10)
num_train_epochs: 10
per_device_train_batch_size: 2
gradient_accumulation_steps: 8
learning_rate: 1e-05
warmup_ratio: 0.05
max_seq_length: 1024
alpha: 0.7
temperature: 2.0
kd_method: topk
top_k: 64

Notes

Prompt format in this repo is question + "\n\nLet's think step by step.\n".
Original Meta Llama license and access requirements still apply to downstream use.
run_config.json and eval/test_summary.json are uploaded alongside the weights when available.