Diffusion Models

If you’re interviewing for a generative AI role, diffusion models are almost guaranteed to come up. In recent years, diffusion-based generators have been spotlighted in AI thanks to breakthroughs like Stable Diffusion and DALL•E. Interviewers know this is a hot technology, so they frequently ask candidates to explain diffusion models and contrast them with older generative frameworks (GANs and VAEs). This question isn’t just about definitions—it’s about showing you understand the landscape of generative models.

It’s common because hiring teams want to ensure you’re up-to-date with current GenAI techniques (diffusion being the latest trend) while still solid on the fundamentals (GANs and VAEs have been around longer). In other words, “Explain diffusion vs. GANs vs. VAEs” is a proxy for “Does this person grasp how modern generative models work and why we’ve evolved new approaches?” Another reason this question keeps popping up because it reveals your depth of understanding. Nearly every ML engineer has heard of GANs or VAEs from courses or projects; diffusion models, however, are newer and a bit more complex.

In this lesson, we’ll break down all those concepts individually. First, we’ll clarify diffusion models in the context of generative AI. Then we’ll discuss why diffusion models have become so relevant (the hype and real-world impact). After that, we’ll dive deeper into how diffusion models work under the hood (with a conceptual diagram to visualize the process). With that foundation, we’ll compare Diffusion vs. GANs vs. VAEs, highlighting key differences and use cases.

What is diffusion?

In generative AI, a diffusion model is a type of model that generates data by progressively denoising random noise to create a sample. That’s a mouthful, so let’s unpack it. The term “diffusion” comes from the idea of particles diffusing (spreading out) in a medium—imagine ink spreading in water, eventually turning the water uniformly colored. In the context of AI, we simulate a diffusion-like process on data: we start with a piece of data (say, an image) and gradually add more and more noise until it becomes pure noise. This is called the forward diffusion process, and it’s a bit like destroying the image step by step.

Formally, the noisy sample at the time step $t$ is computed as:

Where:

• $\mathbf{x}_0$: Original clean sample (e.g., an image)

• $\mathbf{x}_t$: Noisy version of the sample at the timestep $t$ ...