Official model weights for minute-level multi-shot audio-video generation with a distilled DMD generator, paired cross-modal memory, and story-level consistency.
Echo-LongVideo (a.k.a. JoyAI-Echo) is a long-form, multi-shot, audio-video generation model. A cross-modal audio-visual memory bank preserves character appearance and voice timbre consistently across up to five-minute videos, and a post-training pipeline combining memory-based reinforcement learning with distribution matching distillation (DMD) delivers a 7.5× inference speedup without sacrificing quality.
In human evaluation, Echo-LongVideo decisively outperforms HappyOyster (directing mode) on long-form generation and surpasses the short-video specialist Wan 2.6 on human-centric tasks.
This repository hosts the released checkpoint. Inference code is released separately — see the Usage section.
Model Details
Developed by: Echo Team @ Joy Future Academy, JD
Model type: Text-to-(Audio+Video) diffusion transformer, DMD 8-step
Environment: Python 3.11 + PyTorch 2.8 + CUDA 12.8 (see the inference repo's environment.yml / requirements.txt).
3. Write a story prompt
Enhance your prompt first. We provide prompt enhancers — system prompts that expand a short story or idea into well-formed shot prompts: prompts/long_story_writer_system_prompt.md for long, multi-shot video, and prompts/short_story_writer_system_prompt.md for single-shot short video. We strongly recommend running your input through the matching enhancer before inference; un-enhanced prompts tend to produce noticeably weaker results.
Create a JSON file under prompts/. Each file is a single object with a prompts list, where every string is one complete shot. A single string produces one shot; multiple strings produce a multi-shot story, with each new shot conditioned on the previous ones through the paired audio-video memory bank.
Inside each string, write these parts in order:
Part
What to describe
Roles & Subjects
Describe the appearance of all visible people, including age, build, hair, face, wardrobe, and speaking voice timbre when applicable.
Action & Dialogue
What the subject does and speaks.
Style
The overall visual and emotional aesthetic — e.g. realistic motorsport film language, cool daylight, restrained cinematic tension.
Camera Movement
The shot type and framing or movement — e.g. a stable close-up on the face, or a medium shot from the waist up.
Background
The setting and scene details behind the subject.
Sound Effects & BGM
The sounds in the scene and the background music — e.g. room tone, wind, footsteps and fabric, with a soft low music bed under the dialogue or nobackground music
A more convenient prompt-writing workflow will be released as a director agent for everyone to use.
4. Run
python inference.py
Outputs land in inference_result/outputs/<prompt-name>/inference_<timestamp>/.
Hardware
Peak GPU memory is ~46–50 GB at the default 1280 × 736 × 241 frame setting — a single H100/A100 (80 GB) or 48 GB GPU is sufficient. For smaller GPUs, lower resolution or frame count:
GSB user study. Values are the percentage of user preferences.
Aspect (Long Video)
JoyAI-Echo
Tie
HappyOyster (Directing)
Visual aesthetics
63.6%
8.8%
27.6%
Audio quality
81.7%
6.5%
11.8%
Prompt following
80.6%
13.5%
5.9%
IP consistency
59.4%
12.9%
27.7%
Aspect (Short Video)
JoyAI-Echo
Tie
Wan 2.6
Visual aesthetics
58.8%
14.7%
26.5%
Audio quality
32.3%
30.9%
36.8%
Prompt following
33.8%
36.8%
29.4%
Acknowledgements
We gratefully acknowledge LTX-Video for the base video generator and Gemma for the text encoder, along with the broader open-source community.
Citation
If Echo-LongVideo helps your research or products, please cite:
bibtex
1@techreport{echo2026longvideo,
2 title = {Echo-LongVideo: Pushing the Frontier of Long Video Generation},
3 author = {{Echo Team @ Joy Future Academy, JD}},
4 institution = {Joy Future Academy, JD},
5 year = {2026},
6 month = {June},
7 url = {https://github.com/Echo-Team-Joy-Future-Academy-JD/JoyAI-Echo}
8}
License
Released under the LTX-2 Community License Agreement. By downloading or using these weights, you agree to its terms. The bundled Gemma text encoder is governed by Google's separate Gemma license.
