Techniques are disclosed for removing blur from a single image by accumulating a blur kernel estimation across several scale levels of the image and balancing the contributions of the different scales to the estimation depending on the noise level in each observation. In particular, a set of observations can be obtained by applying a set of variable scale filters to a single blurry image at different scale levels. A single blur kernel can be estimated across all scales from the set of observations and used to obtain a single latent sharp image. The estimation at a large scale level is refined using the observations at successively smaller scale levels. The filtered observations may be weighted during the estimation to balance the contributions of each scale to the estimation of the blur kernel. A deblurred digital image is recovered by deconvolving the blurry digital image using the estimated blur kernel.