Beta

Explore
Marketplace
Neural Labs
Chat
Wallet
Docs

Beta

Explore Marketplace Neural Labs Wallet Docs

Build

Manage

My assets Manage wallet Organization

Community

Discord X (Twitter)Telegram

Docs Privacy Policy Terms and Conditions

Proud member of

Proud member of

Docs Cookie Policy

Privacy Policy Terms and Conditions

© 2026 AlphaNeural AI. All rights reserved.

opsd-qwen3-1.7b-singlegpu – AI Model by emorywang | AlphaNeural AI

You can deploy this model and start earning money today!

emorywang/
opsd-qwen3-1.7b-singlegpu

like

0

self-distillation

Qwen/Qwen3-1.7B

Views

No views yet

OPSD-Qwen3-1.7B (Single-GPU H20 Reproduction)

On-Policy Self-Distillation (OPSD) LoRA adapter for Qwen3-1.7B, reproduced on a single NVIDIA H20 96GB.

Training Config

Base model: Qwen/Qwen3-1.7B
Method: OPSD (fixed teacher, LoRA student)
LoRA: r=64, alpha=128, target=[q,k,v,o,gate,up,down]_proj
Trainer: TRL 0.26 GOLD trainer
Steps: 125 (effective batch=8)
Hardware: 1×H20 96GB, SDPA attention
Dataset: siyanzhao/Openthoughts_math_30k_opsd (29,434 examples)

Files

adapter_*.json/safetensors — Final LoRA adapter (step 125)
checkpoint-100/ — Full training state at step 100 (for paper comparison)

Usage

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-1.7B", torch_dtype="bfloat16")
model = PeftModel.from_pretrained(base, "emorywang/opsd-qwen3-1.7b-singlegpu")
tokenizer = AutoTokenizer.from_pretrained("emorywang/opsd-qwen3-1.7b-singlegpu")