We are excited to introduce LingBot-Video, the first open-source large-scale MoE (Mixture-of-Experts) video generation model dedicated to embodied intelligence. As a top-tier video model, LingBot-Video is designed to bridge the gap between video synthesis and physical world understanding.
🔥 Key Highlights
🚀 Efficient MoE Architecture: Scaled from scratch; balanced between capacity and cost with ~3x faster inference.
📦 Data Engine: Trained on massive web videos integrated with 70,000+ hours of embodied data.
⚖️ Multi Reward System: Rewarded for high aesthetics, physical rationality, and task completion.
🔥 Latest News
July 9, 2026: 🎉 We release the technical report, code, models, rewriters for LingBot-Video.
The root requirements.txt includes the recommended PyTorch build for LingBot-Video inference.
bash
1git clone https://github.com/Robbyant/lingbot-video
2cd lingbot-video
34python -m venv .venv
5source .venv/bin/activate
6python -m pip install -U pip
78# Base requirements cover direct DiT inference and rewriter --backend transformers.9pip install -r requirements.txt
10pip install -e .
💡 Rewriter deployment: the bundled rewriter uses the single-process
transformers backend. For higher throughput, deploy the VLM yourself and call
it through an OpenAI-compatible API. Preserve the two-stage semantics: step 1
must use the base VLM without the rewriter LoRA, while step 2 must use the same
base VLM with the rewriter LoRA enabled. This can be implemented with two
endpoints, or with one server that can select the adapter per request. See
vLLM / SGLang official docs.
Install the optional SGLang dependencies only when using SGLang Diffusion or the
fused / FP8 MoE runtime:
LingBot-Video DiT inference is designed to consume structured JSON captions,
not casual natural-language prompts. The recommended public workflow is:
Rewrite the user's plain prompt with
Prompt Rewriter.
For TI2V, pass the same first frame to the rewriter.
Run Auto Negative by
default to prune the negative prompt for this specific caption.
Run the unified inference runner with --prompt_json and select direct
diffusers or SGLang Diffusion through --backend.
Backend choices:
diffusers: direct diffusers reference path.
sglang: SGLang Diffusion path. If the optional SGLang package is not
installed, it automatically falls back to direct diffusers and prints a
warning. Install requirements-sglang.txt to enable the SGLang runtime.
For multi-GPU inference, add --enable_fsdp_inference to shard the base DiT and
refiner DiT on GPU. This reduces GPU memory pressure after loading, but each
rank still constructs the transformer on host memory before FSDP sharding; make
sure the machine has enough system RAM for large MoE checkpoints.
Multi-GPU refiner scripts use CP8 + FSDP + batched CFG by default. They also
default to direct diffusers; set BACKEND=sglang externally when you want to
exercise SGLang Diffusion. MoE multi-GPU T2V/TI2V scripts additionally run the
refiner.
All scripts accept the same environment overrides, such as PROMPT_JSON,
IMAGE, OUT_DIR, HEIGHT, WIDTH, STEPS, GUIDANCE_SCALE, SHIFT,
SEED, FPS, BACKEND, and PYTHON_BIN. Refiner scripts also accept
REFINER_HEIGHT, REFINER_WIDTH, REFINER_STEPS,
REFINER_GUIDANCE_SCALE, REFINER_SHIFT, REFINER_T_THRESH, and
REFINER_SIGMA_TAIL_STEPS. MoE scripts default to grouped expert execution
(LINGBOT_MOE_EXPERT_BACKEND=grouped_mm).
See English Docs or 中文文档 for the
detailed prompt rewrite, auto-negative, TI2V, base-only/refiner, distributed
SGLang, and speed-first FP8 workflows.
Note: Bold indicates the best performance, and underline indicates the second best.
⚖️ License
This project is licensed under the Apache 2.0 License. Please refer to the LICENSE file for the full text, including details on rights and restrictions.
📚 Citation
If you find this work useful for your research, please cite our paper:
bibtex
1@article{lingbot-video,
2 title = {Scaling Mixture-of-Experts Video Pretraining for Embodied Intelligence},
3 author = {Shuailei Ma and Jiaqi Liao and Xinyang Wang and Jingjing Wang and Chaoran Feng and Zijing Hu and Chong Bao and Zichen Xi and Yuqi Gan and Weisen Wang and Yanhong Zeng and Qin Zhao and Zifan Shi and Wei Wu and Hao Ouyang and Qiuyu Wang and Shangzhan Zhang and Jiahao Shao and Yipengjing Sun and Liangxiao Hu and Lunke Pan and Nan Xue and Kecheng Zheng and Yinghao Xu and Xing Zhu and Yujun Shen and Ka Leong Cheng},
4 journal={arXiv preprint arXiv:2607.xxxxx},
5 year = {2026}
6}
