tmax-qwen3-8b-tmax-sft-full-20260513 – AI Model by osieosie | AlphaNeural AI

You can deploy this model and start earning money today!

osieosie/
tmax-qwen3-8b-tmax-sft-full-20260513

like

0

osieosie/tmax-sft-full-20260513-bad-tool-call-filtered

hamishivi/Qwen3-8B

Views

No views yet

sft_qwen3_8b_tmax_full_0513_2node

Supervised fine-tune of hamishivi/Qwen3-8B on osieosie/tmax-sft-full-20260513-bad-tool-call-filtered, mixing all 8 subsets at weight 1.0 each.

Training data

Dataset: osieosie/tmax-sft-full-20260513-bad-tool-call-filtered
Subsets:
`allenai__Sera_4.6_Lite_47000` (mixer weight `1.0`)
`m_a_p__TerminalTraj` (mixer weight `1.0`)
`nvidia__Nemotron_Terminal_Corpus__dataset_adapters` (mixer weight `1.0`)
`nvidia__Nemotron_Terminal_Corpus__skill_based_easy` (mixer weight `1.0`)
`nvidia__Nemotron_Terminal_Corpus__skill_based_medium` (mixer weight `1.0`)
`nvidia__Nemotron_Terminal_Corpus__skill_based_mixed` (mixer weight `1.0`)
`open_thoughts__OpenThoughts_Agent_v1_SFT` (mixer weight `1.0`)
`skill_tax_20260505_2.2k_combined_balanced_thinking_all` (mixer weight `1.0`)
Split: train
Transforms: sft_tulu_tokenize_and_truncate_v1, sft_tulu_filter_v1
Dataset cache hash: 8cd5377c34

Hyperparameters

Field	Value
Base model	`hamishivi/Qwen3-8B`
Local model path	`/gpfs/scrubbed/osey/tmax/models/hamishivi/Qwen3-8B`
Max seq length	`32768`
Sequence-parallel size	`1`
Per-device batch size	`2`
Gradient accumulation	`4`
Effective batch size	`128`
Packing	`true`
Learning rate	`1e-5`
LR scheduler	`linear`
Warmup ratio	`0.1`
Epochs	`2`
Checkpoint every	`250` steps
Seed	`123`
Optimizer kernel	Liger
Mixed precision	`bf16`
Gradient checkpointing	`true`
DeepSpeed config	`stage2_no_offloading_accelerate.conf`

Compute

Nodes: 2
GPUs per node: 8 (total 16)
Slurm job: 117347 on g[007,021]
Run timestamp: 20260516_120450

Tracking

W&B project: tmax-full-sft (run name: sft_qwen3_8b_tmax_full_0513_2node)

Training command

sbatch scripts/slurm/sft/qwen/sft_qwen3_8b_tmax_full_0513_2node.sh