Filtered to diffusion transformers · clear filter
This paper helps us understand how text-to-image diffusion transformers work by analyzing the role of "template tokens" in generating images from text prompts. Practitioners might care because it shows how to improve the efficiency of these models without sacrificing their performance.
This paper develops a system called AlayaWorld that can generate interactive virtual worlds from text, images, or videos, allowing for customizable and evolving environments. Practitioners in areas like game development, virtual reality, or interactive storytelling might care about this research for its potential to streamline the creation of immersive experiences.
This paper develops a method to transform video foundation models' representations into compact, reconstruction-capable, and generation-friendly video latents, which can be used in various generative modeling tasks. Practitioners can use VideoRAE to improve the performance of their models by leveraging the semantic and spatio-temporal structure captured by the frozen video foundation encoder.
This paper introduces a way to control the output of diffusion transformers, a type of AI model used for image generation, by providing it with guidance on specific regions of the image. This can be useful for creative professionals who need to generate images with precise control over details like materials, objects, and spatial arrangements.