THUDM/LongWriter dataset contains 6,000 SFT data with ultra-long output ranging from 2k-32k words in length (both English and Chinese). The data can support training LLMs to extend their maximum output window size to 10,000+ words - used in this model

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.