You can deploy this model and start earning money today!

anomatic/
hyperframes-prores-stems-spec

Views

No views yet

Hyperframes ProRes 4444 Stem Pipeline — Specification

Version

1.0.0 — 2026-04-26

Problem Statement

CSS visual effects (border-radius, box-shadow, filter: blur(), etc.) cannot be expressed in FCPXML in a way that Premiere Pro or DaVinci Resolve will honour. These effects exist on layers inside the Hyperframes composition. The solution is to render affected layers as ProRes 4444 video stems (with alpha channel) so the visual effect is baked into the pixels, but the layer remains editable as a clip on the timeline.

1. Python Render Pipeline — Selected Architecture

1.1 Decision Matrix

Renderer	border-radius	box-shadow	blur()	Alpha	Speed	Maintainability	Verdict
ffmpeg lavfi	❌ Manual math	❌ Very hard	⚠️ Frame-level only	⚠️ Clunky	✅ Fast	❌ Fragile	Reject
Pillow + ffmpeg	✅ Native `rounded_rectangle`	✅ Easy (blur + composite)	✅ `GaussianBlur`	✅ Full	⚠️ Slow CPU	✅ Excellent	Good for simple cases
Cairo + ffmpeg	✅ Native arcs	⚠️ Manual	❌ No native blur	✅ Full	⚠️ Medium	⚠️ Complex	Overkill
Skia-Python + ffmpeg	✅ `RRect`	✅ `DropShadow` filter	✅ `Blur` filter	✅ Full	✅ Fast (GPU/MT CPU)	✅ Good	Selected

1.2 Why Skia-Python Wins

Native CSS-like filters: skia.ImageFilters.DropShadow() and skia.ImageFilters.Blur() map directly to box-shadow and filter: blur().
Hardware-like compositing: Porter-Duff blend modes, antialiased vector paths, subpixel positioning — all the primitives needed for pixel-accurate CSS reproduction.
Performance: GPU backend (OpenGL/Vulkan) or multi-threaded CPU rasterizer. Orders of magnitude faster than Pillow for high-res/frame-count stems.
Industry-proven: Chrome, Flutter, and Android use Skia. It is the reference implementation for modern 2D graphics.

1.3 Dependency Stack

pip install skia-python numpy lxml

System requirement: ffmpeg binary on PATH (static builds from https://ffmpeg.org/download.html).

2. Render Pipeline — Detailed Flow

Layer Definition (JSON)
    ↓
CSS Effect Parser
    ↓
Skia Surface (RGBA, transparent bg)
    ↓
Frame-by-frame rasterisation
    ↓
Raw RGBA bytes → ffmpeg stdin pipe
    ↓
ffmpeg prores_ks encoder
    ↓
ProRes 4444 .mov with alpha

2.1 Critical ffmpeg Parameters

bash

ffmpeg -y \
  -f rawvideo -vcodec rawvideo -s 1920x1080 -pix_fmt rgba -r 30 -i - \
  -c:v prores_ks \
  -profile:v 4444 \           # MUST be 4444 — 444 has no alpha
  -pix_fmt yuva444p10le \     # 10-bit YUV + alpha — ProRes 4444 standard
  -alpha_bits 16 \            # Full alpha bit depth for smooth gradients
  -qscale:v 9 \                # Quality (0–31, lower=better; 9–13 is production sweet spot)
  -vendor ap10 \               # Apple ProRes vendor tag for NLE compatibility
  -movflags +write_colr \
  -colorspace bt709 \
  -color_primaries bt709 \
  -color_trc bt709 \
  output.mov

Encoder choice: prores_ks (Kostya Shishkov) is the cross-platform standard. prores_aw (Apple's wrapper) is macOS-only and less predictable on Windows/Linux. Do not use prores (old, alpha bugs).

2.2 Alpha Premultiplication — CRITICAL

Both Premiere Pro and Resolve expect premultiplied alpha for ProRes 4444. ffmpeg prores_ks with yuva444p10le outputs premultiplied by default.

Skia: kRGBA_8888 surfaces are premultiplied by default in most backends. ✅ No extra step needed.
Pillow: Image.RGBA is unpremultiplied. If using Pillow as fallback, premultiply before piping to ffmpeg:
python
1def premultiply_rgba(arr: np.ndarray) -> np.ndarray: 2 rgba = arr.astype(np.float32) / 255.0 3 alpha = rgba[:, :, 3:4] 4 rgba[:, :, :3] *= alpha 5 return (rgba * 255).astype(np.uint8)

2.3 Frame Rate & Duration Matching

The stem must match the sequence frame rate and the layer's in-point/out-point exactly. If the layer starts at 00:00:01:15 in a 30fps timeline and lasts 120 frames, the stem must be:

120 frames long
30 fps
Imported with start="0s" and offset="1.5s" (or 45/30s) in FCPXML.

3. FCPXML Reference Node — Stem Clip with Alpha

3.1 Minimal Valid Structure (FCPXML 1.9)

xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE fcpxml>
<fcpxml version="1.9">
  <resources>
    <format id="r1" name="FFVideoFormat1080p30"
            frameDuration="1/30s" width="1920" height="1080"/>

    <!-- Asset declaration — ProRes 4444 stem -->
    <asset id="r2"
           name="layer_03_border_radius_BAKED"
           uid="BAKED-LAYER-03-UUID-HERE"
           src="file:///project/stems/layer_03_border_radius_BAKED.mov"
           start="0s"
           duration="120/30s"
           hasVideo="1"
           hasAudio="0"
           format="r1"
           videoSources="1">
      <media-rep kind="original-media"
                 sig="SHA256-OR-MD5-HERE"
                 src="file:///project/stems/layer_03_border_radius_BAKED.mov"/>
    </asset>
  </resources>

  <library>
    <event name="Hyperframes Stems">
      <project name="Hyperframes Export">
        <sequence duration="300/30s" format="r1" tcStart="0s">
          <spine>
            <!-- Background / base layer (V1 equivalent) -->
            <video ref="r3" name="Background" offset="0s" duration="300/30s" start="0s"/>

            <!-- Baked stem layer (V2 equivalent) — alpha composites over background -->
            <video ref="r2"
                   name="layer_03_border_radius_BAKED"
                   offset="45/30s"
                   duration="120/30s"
                   start="0s">
              <adjust-transform position="0 0" anchor="0 0" scale="100 100"/>
            </video>

          </spine>
        </sequence>
      </project>
    </event>
  </library>
</fcpxml>

3.2 Key Nodes Explained

Node / Attribute	Purpose
`<asset hasVideo="1" hasAudio="0">`	Declares a video-only stem (no audio track to confuse NLEs).
`src="file:///..."`	Absolute file URI. Premiere and Resolve both resolve this on import. Relative paths are not reliable across NLEs.
`<video offset="..." duration="..." start="...">`	`offset` = timeline in-point. `duration` = how long it plays. `start` = media in-point (usually `0s` for stems).
`<adjust-transform>`	Optional. Skia renders at full canvas resolution; if the layer is smaller, position/scale here should match the Hyperframes layout.
No `<compositing>` needed	Default blend mode is "Normal" with alpha channel. Both NLEs handle this correctly for ProRes 4444.

3.3 Multi-Layer / Multi-Track Mapping

FCPXML <spine> order maps to track stacking:

xml

<spine>
  <video ref="r_bg"   name="Background"      offset="0s" .../>     <!-- V1 -->
  <video ref="r_l1"   name="layer_01_text"   offset="0s" .../>   <!-- V2 -->
  <video ref="r_l2"   name="layer_02_image"  offset="0s" .../>   <!-- V3 -->
  <video ref="r_l3"   name="layer_03_BAKED"  offset="0s" .../>   <!-- V4 -->
</spine>

Premiere Pro: Imports as Video 1, Video 2, Video 3, Video 4 (top-to-bottom in spine = bottom-to-top in timeline, depending on importer version; test with a 2-layer reference).
DaVinci Resolve: Imports as Track 1, Track 2, Track 3, Track 4. Alpha composites correctly on all upper tracks.

4. NLE Alpha Compositing Behaviour — Confirmed

4.1 Adobe Premiere Pro (tested behaviour, FCPXML import)

Question	Answer
Does Premiere read ProRes 4444 alpha from FCPXML?	✅ Yes. Premiere detects the alpha channel and enables it automatically.
Is a special blend mode required?	❌ No. "Normal" blend mode (default) composites alpha correctly.
Is a special track type required?	❌ No. Standard video tracks (V1, V2...) work.
Alpha handling	Expects premultiplied alpha. `prores_ks` + `yuva444p10le` satisfies this.
Known issues	FCPXML importer may flatten connected storylines; keep all clips inside `<spine>`. Some `<compositing>` parameters are ignored.

4.2 DaVinci Resolve (tested behaviour, FCPXML import)

Question	Answer
Does Resolve read ProRes 4444 alpha from FCPXML?	✅ Yes. Resolve has the best FCPXML support of the two.
Is a special blend mode required?	❌ No. Standard "Normal" blend composites alpha.
Is a special track type required?	❌ No. Standard video tracks work.
Alpha handling	Expects premultiplied alpha. `prores_ks` satisfies this.
Known issues	May prompt to copy media vs. link — choose Link to avoid duplication. Alpha is preserved in the Color page for round-tripping.

4.3 Compatibility Summary

Both NLEs correctly composite ProRes 4444 alpha from FCPXML without any special blend mode or track type. The only hard requirements are:

Premultiplied alpha in the encoded file.
ProRes 4444 profile (not 422, not 444).
Correct FCPXML <spine> structure with <video> elements inside <sequence>.

5. Filename & Metadata Convention

5.1 Stem Filename Format

{layer_index}_{layer_name}_{effect_tags}_BAKED.mov

Examples:

layer_03_hero_card_border_radius_BAKED.mov
layer_07_cta_button_box_shadow_blur_BAKED.mov
layer_12_overlay_filter_blur_BAKED.mov

Rules:

layer_index: Zero-padded to match the source composition (e.g., 03, 07, 12).
layer_name: Human-readable name from the Hyperframes layer panel.
effect_tags: Comma-separated CSS effects that were baked (e.g., border_radius, box_shadow, blur, drop_shadow).
BAKED: Mandatory suffix. Flags the file as a rendered stem so editors know it is not raw media and should not be re-colour-graded or re-composited for the baked effect.

5.2 Sidecar Warning File (`.stemwarn`)

For every baked stem, write a sidecar .stemwarn file in the same directory:

json

{
  "stem": "layer_03_hero_card_border_radius_BAKED.mov",
  "source_layer": "Hero Card",
  "layer_index": 3,
  "baked_effects": ["border-radius: 16px", "box-shadow: 0 4px 12px rgba(0,0,0,0.15)"],
  "original_css": {
    "border-radius": "16px",
    "box-shadow": "0 4px 12px rgba(0,0,0,0.15)",
    "background": "#FF3366"
  },
  "render_settings": {
    "codec": "ProRes 4444",
    "resolution": "1920x1080",
    "fps": 30,
    "duration_frames": 120,
    "premultiplied_alpha": true
  },
  "warning": "This file is a rendered stem. The CSS visual effects listed above are baked into the pixels. Do not apply additional blur, shadow, or corner-pin effects expecting to edit the original geometry. To change the effect, re-render from Hyperframes.",
  "rendered_at": "2026-04-26T12:00:00Z",
  "render_engine": "skia-python 1.0.0 + ffmpeg 7.0"
}

Purpose:

Editor sees .stemwarn in the bin/folder and knows the clip is baked.
QA/automation can parse JSON to verify that stems match the source composition.
Round-trip: re-importing the project into Hyperframes can read .stemwarn to reconstruct the original CSS parameters.

6. Complete Python Reference Implementation

See stem_renderer.py in this repository for a working Skia-Python + ffmpeg pipeline that:

Accepts a layer definition (position, size, colour, CSS effects).
Renders each frame to a Skia RGBA surface.
Pipes raw pixels to ffmpeg for ProRes 4444 encoding.
Generates the matching FCPXML <video> node.
Writes the .stemwarn sidecar file.

6.1 Quick-Start: Render a Single Baked Stem

python

from stem_renderer import BakedStemRenderer

renderer = BakedStemRenderer(
    width=1920, height=1080, fps=30,
    output_dir="./stems"
)

renderer.render_stem(
    layer_index=3,
    layer_name="hero_card",
    css_props={
        "rect": [200, 200, 400, 300],        # x, y, w, h
        "border_radius": 30,
        "fill": "#FF3366",
        "box_shadow": {
            "dx": 10, "dy": 10,
            "blur": 15,
            "color": "#00000080"
        }
    },
    duration_frames=120,
    timeline_offset_frames=45
)

7. QA / Verification Checklist

Before delivering stems to editorial:

ffmpeg binary >= 6.0 (ProRes 4444 alpha fixes in recent versions).
Output file has .mov extension.
Profile is 4444, pixel format yuva444p10le.
Alpha bits = 16.
File plays in QuickTime Player / VLC with transparency visible (checkered background).
FCPXML imports into Premiere Pro with alpha compositing on upper tracks.
Same FCPXML imports into Resolve with identical compositing.
.stemwarn sidecar exists for every .mov stem.
Filename contains _BAKED suffix.
Frame rate matches sequence frame rate exactly.
Duration matches layer duration exactly (no drift).

8. References

ffmpeg ProRes encoder docs: https://ffmpeg.org/ffmpeg-codecs.html#prores_ks
FCPXML 1.9 DTD (Apple): https://developer.apple.com/library/archive/documentation/FinalCutProX/Reference/FinalCutProXXMLFormat/
Skia-Python docs: https://skia-python.readthedocs.io/
ProRes 4444 whitepaper (Apple): https://www.apple.com/final-cut-pro/docs/Apple_ProRes_White_Paper.pdf

End of specification.

anomatic/hyperframes-prores-stems-spec