Flow Matching & Diffusion Prediction Types

A Practical Guide to Sol, Lune, and Epsilon Prediction

Overview

1. Epsilon (ε) Prediction — Standard Diffusion

Core Concept

"Predict the noise that was added"

The Formula (Simplified)

TRAINING:
    x_noisy = √(α) * x_clean + √(1-α) * noise
    ↓
    Model predicts: ε̂ = "what noise was added?"
    ↓
    Loss = ||ε̂ - noise||²

SAMPLING:
    Start with pure noise
    Repeatedly ask: "what noise is in this?"
    Subtract a fraction of predicted noise
    Repeat until clean

Reading the Math

• α (alpha): "How much original image remains" (1 = all original, 0 = all noise)
• √(1-α): "How much noise was mixed in"
• ε: The actual noise that was added
• ε̂: Model's guess of what noise was added

Training Process

Sampling Process

Utility & Behavior

• Strength: General-purpose image generation
• Weakness: No explicit understanding of image structure
• Use case: Standard text-to-image generation

2. Velocity (v) Prediction — Sol (DDPM Framework)

Core Concept

"Predict the direction from noise to data"

The Formula (Simplified)

TRAINING:
    x_t = α * x_clean + σ * noise       (same as DDPM)
    v = α * noise - σ * x_clean         (velocity target)
    ↓
    Model predicts: v̂ = "which way is the image?"
    ↓
    Loss = ||v̂ - v||²

SAMPLING:
    Convert velocity → epsilon
    Use standard DDPM scheduler stepping

Reading the Math

• v (velocity): Direction vector in latent space
• α (alpha): √(α_cumprod) — signal strength
• σ (sigma): √(1 - α_cumprod) — noise strength
• The velocity formula: v = α * ε - σ * x₀
  • "Velocity is the signal-weighted noise minus noise-weighted data"

Why Velocity in DDPM?

• DDPM noise schedule for interpolation
• Velocity prediction for training target
• Knowledge distillation from a teacher

Training Process (David-Weighted)

Sampling Process (CRITICAL: v → ε conversion)

Utility & Behavior

• What Sol learned: Platonic forms, silhouettes, mass distribution
• Visual output: Flat geometric shapes, correct spatial layout, no texture
• Why this happened: David rewarded geometric coherence, Sol optimized for clean David classification
• Use case: Structural guidance, composition anchoring, "what goes where"

Sol's Unique Property

• Castle prompt → Castle silhouette, horizon line, sky gradient
• Portrait prompt → Head oval, shoulder mass, figure-ground separation
• City prompt → Building masses, street perspective, light positions

3. Velocity (v) Prediction — Lune (Rectified Flow)

Core Concept

"Predict the straight-line direction from noise to data"

The Formula (Simplified)

TRAINING:
    x_t = σ * noise + (1-σ) * data      (linear interpolation)
    v = noise - data                     (constant velocity)
    ↓
    Model predicts: v̂ = "straight line to noise"
    ↓
    Loss = ||v̂ - v||²

SAMPLING:
    Start at σ=1 (noise)
    Walk OPPOSITE to velocity (toward data)
    End at σ=0 (clean image)

Reading the Math

• σ (sigma): Interpolation parameter (1 = noise, 0 = data)
• x_t = σ·noise + (1-σ)·data: Linear blend between noise and data
• v = noise - data: The velocity is CONSTANT along the path
• Shift function: σ' = shift·σ / (1 + (shift-1)·σ)
  • Biases sampling toward cleaner images (spends more steps refining)

Key Difference from Sol

Training Process

Sampling Process (Direct Euler)

Why SUBTRACT the Velocity?

v = noise - data  (points FROM data TO noise)

We want to go FROM noise TO data (opposite direction!)

So: x_new = x_current - v * dt
         = x_current - (noise - data) * dt
         = x_current + (data - noise) * dt  ← Moving toward data ✓

Utility & Behavior

• What Lune learned: Rich textures, fine details, realistic rendering
• Visual output: Full detailed images with lighting, materials, depth
• Training focus: Portrait/pose data with caption augmentation
• Use case: High-quality image generation, detail refinement

Comparison Summary

Training Targets

EPSILON (ε):     target = noise
                 "What random noise was added?"

VELOCITY (Sol):  target = α·noise - σ·data  
                 "What's the DDPM-weighted direction?"

VELOCITY (Lune): target = noise - data
                 "What's the straight-line direction?"

Sampling Directions

EPSILON:         x_new = scheduler.step(ε_pred, t, x)
                 Scheduler handles noise removal internally

VELOCITY (Sol):  Convert v → ε, then scheduler.step(ε, t, x)
                 Must translate to epsilon for DDPM math

VELOCITY (Lune): x_new = x - v_pred * dt
                 Direct Euler integration, subtract velocity

Visual Intuition

EPSILON:
    "There's noise hiding the image"
    "I'll predict and remove the noise layer by layer"
    → General-purpose denoising

VELOCITY (Sol):  
    "I know which direction the image is"
    "But I speak through DDPM's noise schedule"
    → Learned structure, outputs skeletons

VELOCITY (Lune):
    "Straight line from noise to image"
    "I'll walk that line step by step"  
    → Learned detail, outputs rich images

Practical Implementation Checklist

For Epsilon Models (Standard SD1.5)

• Use DDPM/DDIM/Euler scheduler
• Pass timestep as integer [0, 999]
• Scheduler handles everything

For Sol (Velocity + DDPM)

• Use DDPMScheduler
• Model outputs velocity, NOT epsilon
• Convert: x0 = (α·x - σ·v) / (α² + σ²), then ε = (x - α·x0) / σ
• Call scheduler.step(ε, t, x)
• Expect geometric/structural output

For Lune (Velocity + Flow)

• NO scheduler needed — direct Euler
• Sigma goes 1 → 0 (not 0 → 1!)
• Apply shift: σ' = shift·σ / (1 + (shift-1)·σ)
• Timestep to model: σ * 1000
• SUBTRACT velocity: x = x - v * dt
• Expect detailed textured output

Why This Matters for TinyFlux

1. Sol (early timesteps): Provides geometric anchoring
   • "Where should the castle be?"
   • "What's the horizon line?"
   • "How is mass distributed?"
2. Lune (mid/late timesteps): Provides detail refinement
   • "What texture is the stone?"
   • "How does light fall?"
   • "What color is the sky?"

Quick Reference Card

┌─────────────────────────────────────────────────────────────┐
│                    PREDICTION TYPES                         │
├─────────────────────────────────────────────────────────────┤
│ EPSILON (ε)                                                 │
│   Train: target = noise                                     │
│   Sample: scheduler.step(ε_pred, t, x)                      │
│   Output: General images                                    │
├─────────────────────────────────────────────────────────────┤
│ VELOCITY - SOL (DDPM framework)                             │
│   Train: target = α·ε - σ·x₀                                │
│   Sample: v→ε conversion, then scheduler.step(ε, t, x)      │
│   Output: Geometric skeletons                               │
├─────────────────────────────────────────────────────────────┤
│ VELOCITY - LUNE (Rectified Flow)                            │
│   Train: target = noise - data                              │
│   Sample: x = x - v·dt  (Euler, σ: 1→0)                     │
│   Output: Detailed textured images                          │
└─────────────────────────────────────────────────────────────┘