LP-Distilled0.1 ONNX
LP-Distilled0.1.onnx is an experimental/proof-of-concept browser-oriented distillation of a
LivePortrait-style portrait animation pipeline. It takes a static portrait crop,
a LivePortrait appearance feature volume, normalized head-pose controls, and
normalized expression controls, then renders a 256x256 RGB animation frame.
This release is intended for interactive portrait-animation research and demos.
It is not a full replacement for the original LivePortrait pipeline and should not be used in production.
Model Details
| Field | Value |
|---|
| Model file | LP-Distilled0.1.onnx |
| Format | ONNX |
| ONNX opset | 17 |
| ONNX IR version | 8 |
| Size | 74,573,121 bytes, about 71.1 MiB |
| SHA-256 | 335F98B44E5FAB3577B15C3EC53EDDD39BE228F072D98D7CE11232BCF1C42AFD |
| Output resolution | 256x256 |
| Teacher | Internal LivePortrait-based teacher pipeline |
| Runtime target | Browser ONNX Runtime Web / WebGPU, with WASM fallback where needed |
The released ONNX graph is a fused two-stage model:
1Stage1: source portrait + feature_3d + pose controls -> posed neutral RGB
2Stage2: Stage1 RGB + same source feature_3d + expression controls -> final RGB
The fused graph routes the Stage1 rgb output into Stage2's image input. The
external caller only has to feed the original aligned portrait, the source
feature_3d, and the two control vectors.
Inputs And Outputs
| Name | Shape | Type | Description |
|---|
feature_3d | [1, 32, 16, 64, 64] | float32 | LivePortrait appearance feature volume extracted once from the aligned source portrait. |
pose_control_vector | [1, 3] | float32 | Normalized [yaw, pitch, roll] controls. |
input_rgb | [1, 3, 256, 256] | float32 | Aligned source portrait crop in CHW RGB format. |
expression_control_vector | [1, 30] | float32 | Normalized expression, blink, eye-drift, mood, and viseme controls. |
rgb | [1, 3, 256, 256] | float32 | Rendered output frame in CHW RGB format. |
The model does not include face detection, crop/alignment, or appearance feature
extraction. A browser demo can prepare those inputs with compatible
LivePortrait-derived ONNX source-preparation models:
1retinaface_det_static.onnx
2face_2dpose_106_static.onnx
3appearance_feature_extractor.onnx
The fused model does not require motion_extractor.onnx.
Control Contract
All controls are normalized numeric values. They are not image-space landmark
coordinates.
Pose controls:
1pose.yaw
2pose.pitch
3pose.roll
Expression controls:
1kp1.x
2kp1.y
3kp1.z
4kp2.x
5kp2.y
6kp2.z
7kp3.y
8kp5.x
9kp5.y
10kp5.z
11kp6.y
12kp6.z
13kp11.x
14kp11.y
15kp13.y
16kp14.x
17kp14.y
18kp15.x
19kp15.y
20kp16.x
21kp16.y
22kp17.x
23kp17.y
24kp17.z
25kp19.x
26kp19.y
27kp19.z
28kp20.x
29kp20.y
30kp20.z
The control schema deliberately excludes many LivePortrait keypoint axes.
Runtime code should use the exact schema order above rather than assuming
keypoint * 3 + axis layout.
Keypoint expression controls are normalized deltas, not absolute keypoints. The
training convention was:
1raw_delta = clamp(raw_delta, -0.08, 0.08)
2normalized_delta = raw_delta / 0.08
Pose controls are also normalized to [-1, 1]. The training convention was:
1yaw_norm = yaw_degrees / 20
2pitch_norm = pitch_degrees / 14
3roll_norm = roll_degrees / 14
Those ranges are part of the model contract. Sending much larger values can
produce degraded or unstable output.
Pose Vocabulary
The pose model was trained primarily around a finite vocabulary of Catmull-Rom
head-pose trajectories, plus single-axis paths. This is intentional: the runtime
motion system is expected to follow, stop, reverse, or blend from learned
trajectories rather than freely sampling every possible yaw/pitch/roll
combination.
Inputs outside the trained trajectory vocabulary may still work, but quality can
degrade, especially for extreme rotations or unusual pose/expression
combinations.
Limitations
- Output is fixed at
256x256.
- This proof-of-concept model was trained on only a few hundred portrait style
images. In adhoc testing it can generalize some, but might not look great,
depending on the specific source image.
- Quality depends heavily on source crop/alignment quality.
- The model can blur or distort hair, facial details, accessories, and unusual
backgrounds.
- Extreme poses can be softer than neutral or moderate poses.
- Controls are semantic approximations learned from a teacher; they are not a
physically meaningful face rig.
- The model card does not assert compatibility with all upstream licenses. Check
the relevant LivePortrait and dataset licenses before redistribution or
commercial use.
Ethical Considerations
Portrait animation models can be misused to create misleading or non-consensual
media. Applications should clearly disclose generated animation, obtain consent
for likeness use, and avoid representing outputs as authentic footage.
Example Runtime Flow
11. Load a portrait image.
22. Detect and align the face to a 256x256 RGB crop.
33. Run a compatible LivePortrait appearance feature extractor once:
4 input_rgb -> feature_3d
54. For each frame:
6 feature_3d
7 pose_control_vector
8 input_rgb
9 expression_control_vector
10 -> LP-Distilled0.1.onnx
11 -> rgb
125. Draw rgb to a canvas or video frame.
For lip-sync, map phonemes or visemes to the 30-wide expression control vector,
then smooth the control curve over time. Directly snapping between discrete
viseme vectors will look jerky.
Release Notes
This is an early research release. The current graph is useful for interactive
demos, but quality is still evolving. Future releases may change the control
schema, model capacity, training data mix, or source-prep assumptions.
