Licon Live Talk
Overview
Licon Live Talk is a digital human generation model built on the LTX native audio-video generation architecture. Given a reference portrait and text guidance, it generates synchronized audio-video content as a continuously extendable stream rather than a fixed-length clip.
- Character identity and visual continuity can be maintained across long sequences.
- Generation can continue beyond a fixed clip length without restarting the character from scratch.
- Native audio-video generation provides a practical foundation for a more natural and interactive digital human.
The current release demonstrates that this technical direction is feasible. It is intended for research, evaluation, and further development, not production deployment.
Inference Requirements
- 4 × NVIDIA H200 GPUs for the core generator.
- At least 1 additional GPU for auxiliary inference components.
- Linux, Python 3.11, and an NVIDIA driver compatible with CUDA 12.6.
The inference runtime, installation steps, and model download instructions are available at:
Self-Forcing Approach
The generator is adapted with a Self-Forcing training strategy. Instead of relying only on clean teacher trajectories, the model is trained to continue from states produced by its own rollout. This reduces the mismatch between training conditions and real inference, where every new segment must build on content generated by the model itself.
Self-Forcing describes the training strategy. For streaming inference, this release adds an anchor-and-history continuation mechanism organized around two forms of generated context:
- Identity anchor: the first generated audio-video latent block from the reference-conditioned opening segment. It remains fixed to help preserve the character's appearance.
- Rolling history: the most recent generated audio-video latent tail, retained to preserve local motion and temporal continuity across segments.
Within each segment, new audio-video blocks are generated causally from the anchor, recent history, and current text guidance, while a temporary KV cache advances block by block. When the segment is complete, its latest latent tail becomes the rolling history for the next segment while the identity anchor remains fixed. This process enables the model's infinite-generation mode while keeping the cross-segment context bounded.
Audio and video are generated within the same LTX-based pipeline. They are not produced as two unrelated outputs and combined only at the end, which provides a stronger base for synchronized character performance.
Current Capabilities
- Maintains character identity and visual continuity across extended sequences.
- Supports continuous, open-ended generation by extending the sequence segment by segment.
- Produces native joint audio-video output through the underlying LTX architecture.
- Supports interactive continuation driven by text messages during an active session.
"Infinite generation" refers to a continuously extendable streaming process with no fixed pre-declared clip length. Practical session length still depends on the deployment environment and application.
Inputs and Output
- Inputs: one reference portrait image and text guidance.
- Output: a continuously streamed audio-video digital human sequence.
- Validated profile: 704 × 704 resolution, 24 fps, BF16 inference, and four denoising steps.
Validation Demo
This validation release focuses on three questions:
Current Limitations
- Character performance can appear rigid.
- The range and variety of character actions are currently limited.
Roadmap
- Scale up formal training with broader and more diverse data.
- Integrate multi-reference training (MSR) to improve character consistency and performance flexibility.
- Add controllable character actions and richer interactive behavior.
License
The inference code is released under the Apache License 2.0. Bundled third-party components retain their original licenses and terms. See THIRD_PARTY_NOTICES.md in the inference repository.
