An image de-blurring system obtains a blurred input image and generates, based on the blurred input image, a blur kernel. The blur kernel is an indication of how the image capture device was moved and/or how the subject captured in the image moved during image capture, resulting in blur. Based on the blur kernel and the blurred input image, a de-blurred image is generated. The blur kernel is generated based on sharp versions of the blurred input image predicted using a data-driven approach based on a collection of prior edges.