You can deploy this model and start earning money today!

aipracticecafe/
toy-diffusion

safetensors

text-to-image

aipracticecafe/anime-faces-256px-v2

mit

Views

No views yet

Toy-diffusion Ablation Results & Benchmark Analysis

Following the results presented in the i1 paper we conducted a series of architectural controlled experiments on our aipracticecafe/anime-faces-256px-v2 dataset (comprising 87,505 high-quality face crops) using Flux 2 VAE latents and using the Toy-diffusion training code.

Both Single and Dual stream dits completely remove the ADA-LN conditioning, instead the timestep is projected as a token and prepended to the text tokens. Training uses AdamW 8bit optimizer with warmup-constant learning rate of 4e-4 and a batch size of 256.

Below is the detailed benchmark performance FID-20K (Fréchet Inception Distance, ) across our evaluated checkpoints.

1. Unified Leaderboard of Evaluated Checkpoints

Checkpoint Name	Configuration & Parameters	Epoch	FID (↓)	Performance Summary
`dual_stream_768_12_trans_2_bs_1024`	Width: 768, Depth: 12, `trans_2` Adapter, BS: 1024	1400	2.4240	Best Overall Configuration. Large-batch training combined with long training yields the best performance.
`dual_stream_768_12_trans_2`	Width: 768, Depth: 12, `trans_2` Adapter, BS: 512	600	3.0635	Original baseline.
`dual_stream_768_12_llm_te_log_normal`	Width: 768, Depth: 12, LLM Text Encoder, Log-Normal	1000	3.1219	Good prompt following which makes the model prone to instabilities for poorly represented tags.
`dual_stream_768_12_modern_cfm`	Width: 768, Depth: 12, `trans_2` Adapter, Contrastive FM	700	3.2707	Modern text adapter and CFM doesn't seem to provide improvements overbaseline as wall time is similar.
`efficient_unet_bs_256`	SnapGen Conv-Attn Hybrid, BS: 256	400	3.4479	CNN-Transformer hybrid. Fast early-stage convergence at the cost of more than double wall time training and more vram consumption.
`single_stream_768_14_trans_2`	Width: 768, Depth: 14, Single-Stream Backbone, BS: 512	600	3.8954	Slower training throughput. Struggles to drop below FID 3.7.
`sprint_dual_stream_768_13_trans_2_fined`	SPRINT (Sparse), Finetuned Dense on Full Sequence	1000	3.9647	Recovers remarkably well during dense finetuning, preserving visual fidelity while cutting pretraining costs.
`dual_stream_768_16_mlp`	Width: 768, Depth: 16, simple MLP text adapter	400	4.2485	Bottlenecked heavily by the weak MLP text adapter.
`sprint_single_stream_768_14_fined`	SPRINT Single-Stream, Finetuned Dense	1000	7.3793	Does not recover completely from sparse training, lagging significantly behind dual-stream variants.
`sprint_dual_stream_768_13_trans_2`	SPRINT (Sparse), No Finetuning (75% drop ratio)	700	13.8281	High sparse degradation. Evaluating on dense inputs without dense finetuning is not recommended.
`sprint_single_stream_768_14_trans_img_2_rope_text`	SPRINT (Sparse), No Finetuning (75% drop ratio)	750	9.7705	Lower fid than dual-stream version.

Architectural Insights

Text adapter

Our results provide clear, empirical validation of the importantece of a strong text encoder adapter:

Comparing dual_stream_768_16_mlp (depth 16, FID 4.57 at 500 epochs) to dual_stream_768_12_trans_2 (depth 12, FID 3.54 at 500 epochs) reveals a clear pattern. Investing capacity into a 2-layer Transformer Text Adapter (trans_2) rather than raw MLP produces a massive visual upgrade.

B. Single-Stream vs. Dual-Stream Backbones

Training Efficiency: Dual-stream blocks completed training epochs in ~3.5 minutes, whereas Single-stream blocks required ~5.1 minutes—under identical VRAM constraints (~23GB). By processing text and image tokens independently, Dual-stream models seem to be more hardware-friendly.
Visual Fidelity: Dual-stream models drop FID significantly faster and settle at a much lower ultimate floor (FID 3.06 vs. Single-stream's 3.69 plateau).
The Overfitting & Cosplay Tradeoff: We observed a distinct qualitative divergence:
- Dual-stream models are prone to localized concept overfitting (e.g., struggling to render Asuka in Misato's uniform, consistently over-associating characters with their native outfits).
- Single-stream models handle complex conditional blending and "cosplay" prompts with far better zero-shot composition, indicating stronger cross-modality token integration.

C. SPRINT: Efficient Sparse Pre-Training

The sparse token-dropping framework (SPRINT) with a 75% drop ratio achieved a ~50% reduction in training time and lowered active GPU memory requirements from 23GB to 17GB. Even with the Finetuning stage, the overall training wall time is shorter than the dense version.
Finetuning recovers performance: Evaluating SPRINT models directly results in highly degraded FID scores (FID ~13.8) due to the dense-sparse distribution shift. However, following the dense finetuning recipe (_fined), the model recovers rapidly, achieving an FID of 3.96 by epoch 1000.
Dual Stream is favored: Despite the good FID scores exhibited by the Single stream architecture, during the finetuning stage only the dual stream architecture is able to recover the high performance.

Developed by: aipracticecafe

Content Warning & Rights

NSFW Content: This dataset contains sensitive and NSFW (Not Safe For Work) material. It is intended for research and generative modeling purposes.
Legal Disclaimer: I do not own the rights to any of the images in this dataset. All images are the property of their respective creators and were scraped from Danbooru. This dataset is provided for educational and research purposes under fair use.

aipracticecafe/toy-diffusion