Researchers published a study on training pixel-space text-to-image diffusion models. They found large-scale pre-training in pixel space converges far more slowly than in latent space. They propose a latent-to-pixel strategy to acquire priors in latent space first. The study explores weight initialization, data composition, prediction target, decoder architecture and noise schedule. It identifies a practical recipe for pixel-space models to match or exceed latent-space performance. The approach delivers major inference speedups. The paper aims to guide future research on pixel-space generation.
What changed
Latent-space diffusion models were the dominant practical approach for large-scale text-to-image generation.
What it unlocks
Training pixel-space text-to-image diffusion models that match or outperform latent-space models.
- 3.18 to 4.75 times end-to-end inference speedup
Sources