Deep learning models, such as diffusion models, can synthesize images from noise. Diffusion models implement a complex denoising process involving many denoising operations. It can be a challenge to understand the mechanics of diffusion models. To better understand how and when structure is formed, saliency maps and concept formation intensity can be extracted from the sampling network of a diffusion model. Using the input map and the output map of a given denoising operation in a sampling network, a noise gradient map representative of the predicted noise of a given denoising operation can be determined. The noise gradient maps from the denoising operations at different indices can be combined to generate a saliency map. A concept formation intensity value can be determined from a noise gradient map. Concept formation intensity values from the denoising operations at different indices can be plotted.