output_14b_phase2_8k
Qwen3-14B fine-tuned for 3D mesh understanding & generation (SPAC pipeline, Spatial-PE enabled), full fine-tune phase 2 from output_14b_phase1_full.
Training data (bpe_20k_mix, 20k samples + ultrachat_1k, 1k conversations)
- 10k mesh -> text: given a mesh-token sequence, produce a text description (understanding).
- 10k text -> mesh: given a caption + "Reconstruct this 3D shape in mesh token format:", output the mesh token sequence (generation).
Both directions are paired on the same shapes.
Training details
- Full fine-tune (all params), DeepSpeed ZeRO-3, bf16, FlashAttention-2, packing (8192, neat), 8 GPUs, batch 1 each.
- LR 5e-5 cosine decay to 5e-6, warmup 3%.
-
- Epochs: 8.0, train loss: 1.0862, runtime: 22.8 h.
Inference notes
- Chat template:
qwen3_nothink (no thinking). Mesh shapes are tokenized as <mesh_start><morton_*><mesh_*> tokens (see added_tokens.json).
- Requires the SPAC Spatial-PE inference wrapper (
spatial_pe_scale 0.08, merge table from Jack04810/spac-qwen3-4b-spatial-pe-8ep) to run with the spatial positional encoding used in training.
