You can deploy this model and start earning money today!

wangkanai/
wan22-fp16-encoders

Name: wan22-fp16-encoders
Availability: InStock

Views

No views yet

WAN2.2 FP16 Text Encoders

High-precision FP16 text encoders for the WAN (Worldly Advanced Network) 2.2 text-to-video generation system. This repository contains the essential text encoding components required for WAN2.2 video generation workflows.

Model Description

This repository provides two specialized text encoder models optimized for video generation tasks:

T5-XXL FP16: Google's T5 (Text-to-Text Transfer Transformer) extra-extra-large encoder in 16-bit floating point precision
UMT5-XXL FP16: Universal Multilingual T5 extra-extra-large encoder in 16-bit floating point precision

These encoders are critical components of the WAN2.2 pipeline, responsible for transforming text prompts into high-dimensional semantic representations that guide the video generation process. The FP16 precision maintains excellent quality while reducing memory requirements compared to FP32 variants.

Key Features

High Precision: FP16 format preserves text encoding quality with 50% memory reduction vs FP32
Multilingual Support: UMT5-XXL provides robust multilingual text understanding
Production Ready: Optimized for inference with safetensors format
WAN2.2 Compatible: Designed specifically for WAN video generation workflows

Repository Contents

wan22-fp16-encoders/
└── text_encoders/
    ├── t5-xxl-fp16.safetensors       # 8.9 GB
    └── umt5-xxl-fp16.safetensors     # 11 GB

File Details

File	Size	Description
`t5-xxl-fp16.safetensors`	8.9 GB	T5-XXL text encoder (FP16)
`umt5-xxl-fp16.safetensors`	11 GB	Universal Multilingual T5-XXL encoder (FP16)

Total Repository Size: ~20 GB

Hardware Requirements

Minimum Requirements

VRAM: 12 GB GPU memory (for text encoding alone)
RAM: 16 GB system memory
Disk Space: 25 GB free space (including working directory)
GPU: CUDA-compatible GPU with compute capability 6.0+

Recommended Requirements

VRAM: 16 GB+ GPU memory (for full WAN2.2 pipeline)
RAM: 32 GB system memory
Disk Space: 50 GB+ free space
GPU: NVIDIA RTX 3090, RTX 4090, or A100

Performance Notes

Both encoders can be loaded simultaneously with 24 GB+ VRAM
Text encoding typically takes 1-5 seconds per prompt
CPU offloading available but significantly slower (10-30x)

Usage Examples

Basic Text Encoding with Diffusers

python

from diffusers import DiffusionPipeline
import torch

# Load WAN2.2 pipeline with custom text encoders
pipe = DiffusionPipeline.from_pretrained(
    "your-wan22-model",
    text_encoder_path="E:/huggingface/wan22-fp16-encoders/text_encoders/t5-xxl-fp16.safetensors",
    torch_dtype=torch.float16,
    variant="fp16"
).to("cuda")

# Generate video from text
prompt = "A serene mountain landscape at sunset with flowing clouds"
video = pipe(prompt, num_frames=24, height=512, width=512).frames

# Save output
video[0].save("output_video.mp4")

Using UMT5 for Multilingual Prompts

python

from diffusers import DiffusionPipeline
import torch

# Load with multilingual encoder
pipe = DiffusionPipeline.from_pretrained(
    "your-wan22-model",
    text_encoder_path="E:/huggingface/wan22-fp16-encoders/text_encoders/umt5-xxl-fp16.safetensors",
    torch_dtype=torch.float16,
).to("cuda")

# Generate with multilingual prompt
prompt = "東京の夜景、ネオンライトと雨"  # Japanese: Tokyo nightscape with neon lights and rain
video = pipe(prompt, num_frames=48, height=768, width=768).frames

Memory-Optimized Loading

python

from diffusers import DiffusionPipeline
import torch

# Enable CPU offloading for lower VRAM systems
pipe = DiffusionPipeline.from_pretrained(
    "your-wan22-model",
    text_encoder_path="E:/huggingface/wan22-fp16-encoders/text_encoders/t5-xxl-fp16.safetensors",
    torch_dtype=torch.float16,
).to("cuda")

# Enable model CPU offload
pipe.enable_model_cpu_offload()

# Enable attention slicing for further memory reduction
pipe.enable_attention_slicing(1)

# Generate with reduced memory footprint
video = pipe(prompt, num_frames=16).frames

Direct Encoder Loading

python

from safetensors.torch import load_file
import torch

# Load encoder weights directly
encoder_weights = load_file(
    "E:/huggingface/wan22-fp16-encoders/text_encoders/t5-xxl-fp16.safetensors"
)

# Initialize your custom text encoder model
from transformers import T5EncoderModel

text_encoder = T5EncoderModel.from_pretrained("google/t5-v1_1-xxl", torch_dtype=torch.float16)
text_encoder.load_state_dict(encoder_weights)
text_encoder = text_encoder.to("cuda")

# Use encoder for custom workflows
# ... your custom inference code ...

Model Specifications

T5-XXL FP16

Architecture: T5 (Text-to-Text Transfer Transformer)
Model Size: Extra-Extra-Large (XXL)
Parameters: ~11 billion
Precision: FP16 (16-bit floating point)
Format: SafeTensors
Context Length: 512 tokens
Embedding Dimension: 4096
Language Support: English-focused, trained on C4 dataset

UMT5-XXL FP16

Architecture: Universal Multilingual T5
Model Size: Extra-Extra-Large (XXL)
Parameters: ~13 billion
Precision: FP16 (16-bit floating point)
Format: SafeTensors
Context Length: 512 tokens
Embedding Dimension: 4096
Language Support: 100+ languages (multilingual mC4 dataset)

Performance Tips and Optimization

Memory Optimization

Sequential Encoder Loading: Load encoders one at a time if VRAM is limited
CPU Offloading: Use enable_model_cpu_offload() for systems with <16 GB VRAM
Attention Slicing: Enable with enable_attention_slicing() to reduce peak memory
Batch Size: Process multiple prompts together for better GPU utilization

Speed Optimization

TensorRT Compilation: Convert encoders to TensorRT for 2-3x speedup
Flash Attention: Use xformers or flash-attention for faster inference
Model Quantization: Consider INT8 quantization for production deployment
Prompt Caching: Cache encoded prompts for repeated generations

Quality Optimization

Use UMT5 for Non-English: Better results with non-English prompts
Longer Prompts: These XXL models handle detailed descriptions well
Prompt Engineering: Structured, descriptive prompts yield best results
Negative Prompts: Combine with negative prompt encoding for better control

Best Practices

python

# Optimal configuration for quality and speed
pipe = DiffusionPipeline.from_pretrained(
    "wan22-model",
    text_encoder_path="E:/huggingface/wan22-fp16-encoders/text_encoders/t5-xxl-fp16.safetensors",
    torch_dtype=torch.float16,
    variant="fp16",
)

# Enable optimizations
pipe.enable_xformers_memory_efficient_attention()  # Flash attention
pipe.enable_attention_slicing(1)                   # Memory efficiency
pipe.to("cuda")

# Use compiled model for production (PyTorch 2.0+)
pipe.unet = torch.compile(pipe.unet, mode="reduce-overhead", fullgraph=True)

License

These text encoder models are provided under specific licensing terms. Please refer to the original model sources for detailed license information:

T5-XXL: Apache 2.0 License (Google Research)
UMT5-XXL: Apache 2.0 License (Google Research)
WAN2.2 Pipeline: Please check WAN project license terms

Usage Restrictions: These models are intended for research and development purposes. Commercial usage should comply with respective license terms and any additional WAN project requirements.

Citation

If you use these text encoders in your research or projects, please cite the relevant papers:

T5 Citation

bibtex

@article{raffel2020exploring,
  title={Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer},
  author={Raffel, Colin and Shazeer, Noam and Roberts, Adam and Lee, Katherine and Narang, Sharan and Matena, Michael and Zhou, Yanqi and Li, Wei and Liu, Peter J},
  journal={Journal of Machine Learning Research},
  volume={21},
  number={140},
  pages={1--67},
  year={2020}
}

mT5/UMT5 Citation

bibtex

@inproceedings{xue2021mt5,
  title={mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer},
  author={Xue, Linting and Constant, Noah and Roberts, Adam and Kale, Mihir and Al-Rfou, Rami and Siddhant, Aditya and Barua, Aditya and Raffel, Colin},
  booktitle={Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies},
  pages={483--498},
  year={2021}
}

WAN Project Citation

Please check the official WAN project repository for citation guidelines and additional references.

Related Resources

Official Documentation

WAN Project Resources

WAN Official Repository: [Check official project page]
WAN Documentation: [Check official documentation]
Community Forum: [Check community channels]

Related Models

WAN2.2 Base Models: E:/huggingface/wan22/
WAN2.2 VAE: E:/huggingface/wan22-vae/
Enhancement LoRAs: E:/huggingface/wan22-loras/

Technical Support

For issues specific to these text encoders:

Check text encoder dimensions and compatibility with your WAN2.2 version
Verify CUDA and PyTorch versions support FP16 operations
Ensure sufficient VRAM for your chosen encoder(s)
Review memory optimization strategies above

For WAN2.2 pipeline issues, please consult the main WAN project documentation and community resources.

Model Version: v1.2 Last Updated: 2025-10-28 Format: SafeTensors FP16 Compatibility: WAN2.2, Diffusers 0.21+, PyTorch 2.0+

wangkanai/wan22-fp16-encoders

WAN2.2 FP16 Text Encoders

Model Description

Key Features

Repository Contents

File Details

Hardware Requirements

Minimum Requirements

Recommended Requirements

Performance Notes

Usage Examples

Basic Text Encoding with Diffusers

Using UMT5 for Multilingual Prompts

Memory-Optimized Loading

Direct Encoder Loading

Model Specifications

T5-XXL FP16

UMT5-XXL FP16

Performance Tips and Optimization

Memory Optimization

Speed Optimization

Quality Optimization

Best Practices

License

Citation

T5 Citation

mT5/UMT5 Citation

WAN Project Citation

Related Resources

Official Documentation

WAN Project Resources

Related Models

Technical Support

wangkanai/
wan22-fp16-encoders