from safetensors.torch import load_file

# Load just the weights (fast)
state_dict = load_file("weights/lyra_illustrious_best.safetensors")

# Or specific step
state_dict = load_file("weights/lyra_illustrious_step_5000.safetensors")

T5 Input Format

T5 receives a different input than CLIP to enable richer semantic understanding:

CLIP sees:  "masterpiece, 1girl, blue hair, school uniform, smile"
T5 sees:    "masterpiece, 1girl, blue hair, school uniform, smile ¶ A cheerful schoolgirl with blue hair smiling warmly"

The pilcrow (¶) separator acts as a mode-switch token.

Learned Parameters

Alpha (Visibility):

clip_g: 0.7316
clip_l: 0.7316
t5_xl_g: 0.7339
t5_xl_l: 0.7451

Beta (Capacity):

clip_l_t5_xl_l: 0.5709
clip_g_t5_xl_g: 0.5763

Usage

python

from lyra_xl_multimodal import load_lyra_from_hub

model = load_lyra_from_hub("AbstractPhil/vae-lyra-xl-adaptive-cantor-illustrious")
model.eval()

inputs = {
    "clip_l": clip_l_embeddings,     # [batch, 77, 768]
    "clip_g": clip_g_embeddings,     # [batch, 77, 1280]
    "t5_xl_l": t5_xl_embeddings,     # [batch, 512, 2048]
    "t5_xl_g": t5_xl_embeddings      # [batch, 512, 2048]
}

recons, mu, logvar, _ = model(inputs, target_modalities=["clip_l", "clip_g"])

Files

model.pt - Full checkpoint (model + optimizer + scheduler)
checkpoint_lyra_illustrious_XXXX.pt - Step checkpoints
config.json - Training configuration
weights/lyra_illustrious_best.safetensors - Best model weights only
weights/lyra_illustrious_step_XXXX.safetensors - Step checkpoints (weights only)