You can deploy this model and start earning money today!

ZhuofengLi/
qwen3.5-9b-nemotron-sft-ckpt300

Name: qwen3.5-9b-nemotron-sft-ckpt300
Availability: InStock

nvidia/Nemotron-Terminal-Corpus

Qwen/Qwen2.5-VL-7B-Instruct

finetune

apache-2.0

Views

No views yet

Qwen3.5-9B Nemotron SFT Checkpoint-300

Model Description

Intermediate SFT checkpoint (step 300/386, epoch ~1.55/2) of Qwen3.5-9B fine-tuned on the NVIDIA Nemotron-Terminal-Corpus dataset using ms-swift with DeepSpeed ZeRO-3. Trained to improve terminal interaction and agentic capabilities.

Training Data

Dataset: nvidia/Nemotron-Terminal-Corpus
Description: 366k multi-step terminal execution trajectories covering math, code, and software engineering tasks.

Training Details

Framework

Training Framework: ms-swift
Backend: DeepSpeed ZeRO-3

Hyperparameters

Parameter	Value
Learning Rate	2e-5
Weight Decay	1e-4
Epochs	2
Global Batch Size	64 (per_device=1, grad_accum=2)
Micro-batch Size per GPU	1
Max Sequence Length	262144 tokens
Optimizer	AdamW (β₁=0.9, β₂=0.95)
LR Scheduler	Cosine with 10% warmup
Gradient Clipping	1.0
Packing	Enabled
Precision	BF16

Hardware

Item	Value
GPU	H200
Nodes	8
GPUs per Node	8
Total GPUs	64

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(ZhuofengLi/qwen3.5-9b-nemotron-sft-ckpt300, torch_dtype=auto)
tokenizer = AutoTokenizer.from_pretrained(ZhuofengLi/qwen3.5-9b-nemotron-sft-ckpt300)