WAN 2.1 FP8 480p - Image-to-Video Generation Model
This repository contains the WAN (Wan An) 2.1 image-to-video generation model in FP8 precision, optimized for 480p video generation. The FP8 E4M3FN quantization provides approximately 50% memory savings compared to FP16 while maintaining high-quality video generation capabilities.
Model Description
WAN 2.1 FP8 480p is a 14-billion parameter transformer-based diffusion model that transforms static images into dynamic videos. This quantized version offers significant memory efficiency, making it ideal for systems with VRAM constraints or batch processing workflows. The model supports advanced camera control through compatible LoRA adapters (available separately).
Key Capabilities:
Image-to-video generation at 480p resolution
FP8 quantization for efficient inference (~40% VRAM savings)
Compatible with camera control LoRAs for cinematic movements
Fast generation speed on modern GPUs with FP8 support
Note: This repository contains only the diffusion model. For complete functionality, you will need:
WAN 2.1 VAE (243 MB) - Available separately in wan21-vae repository
Camera Control LoRAs (343 MB each) - Optional, available in wan21-loras repository
Hardware Requirements
VRAM: 18GB+ recommended (tested on RTX 4090, RTX 3090)
Disk Space: 16 GB for model file
System RAM: 32GB+ recommended for optimal performance
GPU: NVIDIA GPU with FP8 support recommended (Ada Lovelace/Hopper architecture)
RTX 40 series (4090, 4080): Optimal performance with native FP8
RTX 30 series (3090, 3080): Compatible (falls back to FP16 internally)
Older GPUs: Will work but lose FP8 memory benefits
Usage Examples
Basic Image-to-Video Generation
python
1from diffusers import DiffusionPipeline, AutoencoderKL
2from PIL import Image
3import torch
45# Load the 480p FP8 model6pipe = DiffusionPipeline.from_single_file(7"E:/huggingface/wan21-fp8-480p/diffusion_models/wan/wan21-i2v-480p-14b-fp8-e4m3fn.safetensors",8 torch_dtype=torch.float8_e4m3fn,# FP8 precision9 use_safetensors=True10)1112# Load WAN 2.1 VAE (required, from separate repository)13pipe.vae = AutoencoderKL.from_single_file(14"E:/huggingface/wan21-vae/vae/wan/wan21-vae.safetensors"15)1617pipe.to("cuda")1819# Load input image20input_image = Image.open("path/to/your/image.jpg")2122# Generate video from image23video = pipe(24 image=input_image,25 prompt="cinematic movement, smooth camera motion",26 num_frames=24,27 num_inference_steps=50,28 guidance_scale=7.529).frames[0]3031# Save video32from diffusers.utils import export_to_video
33export_to_video(video,"output_video.mp4", fps=8)
Image-to-Video with Camera Control LoRA
python
1# Load camera control LoRA (from separate repository)2pipe.load_lora_weights(3"E:/huggingface/wan21-loras/loras/wan/wan21-camera-rotation-rank16-v1.safetensors"4)56# Generate video with controlled camera movement7video = pipe(8 image=input_image,9 prompt="rotating camera around the subject, smooth orbital motion",10 num_frames=24,11 num_inference_steps=50,12 guidance_scale=7.513).frames[0]1415export_to_video(video,"output_rotating.mp4", fps=8)
Memory-Optimized Generation
python
1# Enable memory optimizations for lower VRAM usage2pipe.enable_attention_slicing()3pipe.enable_vae_slicing()45# Optional: Enable xformers for faster inference6pipe.enable_xformers_memory_efficient_attention()78# Generate with optimizations active9video = pipe(10 image=input_image,11 prompt="your prompt here",12 num_frames=16,# Reduce frames for lower memory13 num_inference_steps=40# Fewer steps = faster generation14).frames[0]
Model Specifications
Specification
Details
Architecture
Transformer-based image-to-video diffusion model
Parameters
14 billion
Precision
FP8 E4M3FN (8-bit floating point)
Output Resolution
480p
Format
SafeTensors
Quantization
~50% size reduction from FP16
Quality Retention
>95% compared to FP16 variant
Compatible Library
diffusers (requires FP8 support)
Performance Tips
GPU Selection: Best performance on RTX 40 series GPUs with native FP8 support (4090, 4080, 4070 Ti)
Memory Optimization: Use attention slicing and VAE slicing for lower VRAM usage
Frame Count: Start with 16-24 frames for optimal quality/speed balance
Inference Steps: 40-50 steps provide good quality; reduce to 30 for faster generation
Guidance Scale: 7.0-8.0 works well for most prompts; adjust based on desired adherence
Batch Processing: FP8 enables efficient batch processing on 24GB+ GPUs
Faster inference on supported GPUs (RTX 40 series)
Enables larger batch sizes or longer video generation
Compatibility:
Native FP8: RTX 40 series (Ada Lovelace), H100 (Hopper)
Fallback to FP16: RTX 30 series and older (loses memory benefits)
Installation Requirements
bash
1# Install required dependencies2pip install torch>=2.1.0 torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
34# Install diffusers and related packages5pip install diffusers transformers accelerate safetensors
67# Optional: Install xformers for memory-efficient attention8pip install xformers
Minimum Versions:
Python 3.8+
PyTorch 2.1+ (for FP8 support)
diffusers 0.21+
transformers 4.30+
accelerate 0.20+
safetensors 0.3+
Related Models
Same Family:
wan21-fp8-720p - 720p variant (16GB) for higher resolution output
wan21-fp16-480p - FP16 variant (32GB) for maximum precision
wan21-fp16-720p - FP16 720p variant (32GB) for highest quality
Required Components:
wan21-vae - WAN 2.1 VAE (243 MB, required for all WAN 2.1 models)
wan21-loras - Camera control LoRAs (optional, 343 MB each)
Enhanced Version:
wan22-fp8 - WAN 2.2 with enhanced camera controls and quality improvements
Version Information
Version: v1.0 (2024)
Initial release of WAN 2.1 FP8 480p model
FP8 E4M3FN quantization for efficient inference
Compatible with WAN 2.1 VAE and v1 camera control LoRAs
License
This model is released under the WAN license. Please refer to the official WAN model documentation for specific license terms and usage restrictions. Commercial use may have additional requirements.
Citation
If you use this model in your research or projects, please cite:
FP8 Hardware: Best performance requires RTX 40 series or newer; older GPUs fall back to FP16
Resolution: Limited to 480p output; use 720p variant for higher resolution
VAE Dependency: Requires separate WAN 2.1 VAE model for functionality
LoRA Compatibility: Works with WAN 2.1 v1 LoRAs; WAN 2.2 LoRAs may have compatibility issues
Minor Quality Differences: Slight quality variations vs FP16 in extreme lighting/motion scenarios
Support and Resources
Official WAN Documentation: Refer to official WAN model repositories
Community: Hugging Face diffusers community forums
Issues: Report technical issues to the diffusers GitHub repository
Changelog
v1.0 (Initial Release)
WAN 2.1 FP8 480p model release
14B parameters in FP8 E4M3FN precision
Optimized for efficient 480p image-to-video generation
Compatible with WAN 2.1 ecosystem (VAE, LoRAs)
Responsible AI Notice: This model generates video content from images. Please use responsibly and in accordance with ethical AI guidelines. Do not use for creating misleading, harmful, or deceptive content. Consider potential misuse scenarios and implement appropriate safeguards in your applications.
