You can deploy this model and start earning money today!

ZhuofengLi/
Qwen3.5-9B-Base-Nemotron-SFT-5166-steps

Name: Qwen3.5-9B-Base-Nemotron-SFT-5166-steps
Availability: InStock

nvidia/Nemotron-Terminal-Corpus

Qwen/Qwen3.5-9B-Base

finetune

apache-2.0

Views

No views yet

Qwen3.5-9B-Base-Nemotron-SFT-5166-steps

Model Description

This model is Qwen/Qwen3.5-9B-Base fine-tuned with full-parameter SFT on the NVIDIA Nemotron-Terminal-Corpus, following the training recipe from the paper On Data Engineering for Scaling LLM Terminal Capabilities (NVIDIA, 2026). This checkpoint corresponds to step 5166 (final checkpoint after 2 epochs).

Training Data

Dataset: nvidia/Nemotron-Terminal-Corpus
Description: ~366k terminal/coding instruction samples curated by NVIDIA to improve LLM capability on shell, CLI, and code tasks.

Training Details

Framework

Training Framework: ms-swift
Backend: DeepSpeed ZeRO-3

Hyperparameters

Parameter	Value
Learning Rate	2e-5
Weight Decay	1e-4
Epochs	2
Global Batch Size	128
Micro-batch Size per GPU	1
Gradient Accumulation Steps	2
Max Sequence Length	32,768 tokens
Optimizer	AdamW (β₁=0.9, β₂=0.95)
LR Scheduler	Cosine with 10% warmup
Gradient Clipping	1.0
Attention	FlashAttention-2
dtype	bfloat16

Hardware

Item	Value
GPU	H200
Nodes	8
GPUs per Node	8
Total GPUs	64

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "ZhuofengLi/Qwen3.5-9B-Base-Nemotron-SFT-5166-steps",
    torch_dtype="auto",
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("ZhuofengLi/Qwen3.5-9B-Base-Nemotron-SFT-5166-steps")