A diffusion model is augmented with pseudoinverse guidance to restore data, removing artifacts and generating high-quality reconstructed data from limited, low-quality and/or noisy input data. The low-quality input data is denoised by a diffusion model and the denoised input data is combined with a guidance term to produce output data of higher-quality compared with the low-quality input data. The guidance term is a vector-Jacobian product that encourages consistency between the denoised input data and measurements after a pseudoinverse transformation. The denoising process may be applied in an iterative fashion to generate valid solutions to the inverse problem. The augmented diffusion model is a problem-agnostic (e.g., plug-and-play) denoiser that can restore data for a variety of tasks. Example image restoration tasks include denoising, JPEG denoising, deblurring, outpainting, inpainting, colorization, high-dynamic range, and super-resolution.