You can deploy this model and start earning money today!

MUG-V/
MUG-V-training

Name: MUG-V-training
Availability: InStock

Views

No views yet

MUG-V 10B Training Checkpoints

Pre-trained Megatron-format checkpoints for MUG-V 10B video generation model.

Available Checkpoints

MUG-V-10B-torch_dist (Recommended)

Torch Distributed Checkpoint - Flexible parallelism support

Format: Torch Distributed (.distcp)
Parallelism: Can be loaded with any TP/PP configuration
Use Case: Production training, flexible distributed setup

huggingface-cli download MUG-V/MUG-V-training --local-dir ./checkpoints --include "MUG-V-10B-torch_dist/*"

MUG-V-10B-TP4-legacy

Torch Format (Legacy) - Fixed TP=4

Format: Torch format (mp_rank_XX/model_optim_rng.pt)
Parallelism: Must be loaded with TP=4
Use Case: Fixed TP setup or conversion to Torch Distributed

huggingface-cli download MUG-V/MUG-V-training --local-dir ./checkpoints --include "MUG-V-10B-TP4-legacy/*"

Quick Start

Option 1: Direct Training

Use the Torch Distributed checkpoint directly for training:

bash

# Download checkpoint
huggingface-cli download MUG-V/MUG-V-training --local-dir ./checkpoints --include "MUG-V-10B-torch_dist/*"

# Download sample data
huggingface-cli download MUG-V/MUG-V-Training-Samples --repo-type dataset --local-dir ./sample_dataset

# Set environment variables
export CHECKPOINT_DIR="./checkpoints/MUG-V-10B-torch_dist/torch_dist"
export MODEL_TYPE="mugdit_10b"
export DATA_TRAIN="./sample_dataset/train.csv"

# Start training (8 GPUs)
bash examples/mugv/pretrain_slurm.sh

Option 2: Convert to HuggingFace Format

Convert Megatron checkpoint to HuggingFace format for inference:

bash

python -m examples.mugv.convertor.mugdit_mcore2hf \
    --dcp-dir ./checkpoints/MUG-V-10B-torch_dist/torch_dist/iter_0000000 \
    --output ./mugdit_10b_hf.pt \
    --model-size 10B

Checkpoint Formats Comparison

Format	Parallelism	File Structure	Training	Conversion
Torch Distributed	Flexible TP/PP	`*.distcp` files	✅ Recommended	✅ To HF
Torch (Legacy)	Fixed TP=4	`mp_rank_XX/` dirs	⚠️ TP=4 only	✅ To Torch Dist / HF
HuggingFace	None (inference)	Single `.pt` file	❌ Not for training	-

Model Architecture

Parameters: ~10 billion
Architecture: Diffusion Transformer (DiT)
Hidden Size: 3456
Attention Heads: 48
Layers: 56
Compression: VideoVAE 8×8×8

Related Resources

Training Code: MUG-V-Megatron-LM-Training
Inference Code: MUG-V
Inference Weights: MUG-V-inference
Sample Dataset: MUG-V-Training-Samples

Documentation

Training Guide: examples/mugv/README.md
Checkpoint Conversion: Conversion Guide

Citation

bibtex

@article{zhang2025mugv10b,
  title={MUG-V 10B: High-efficiency Training Pipeline for Large Video Generation Models},
  author={Zhang, Yongshun and Fan, Zhongyi and Zhang, Yonghang and Li, Zhangzikang and Chen, Weifeng and Feng, Zhongwei and Wang, Chaoyue and Hou, Peng and Zeng, Anxiang},
  journal={arXiv preprint},
  year={2025}
}

License

Apache License 2.0

Developed by Shopee Multimodal Understanding and Generation (MUG) Team