Methods and systems for a regression-based learning model in image upscaling are disclosed. In one embodiment, a set of image patch pairs for each of a set of images is generated. Each of the image patch pairs contains a natural image and a corresponding downscaled lower-resolution image. A regression model based at least in part on the set of image patch pairs is defined. The regression model represents a gradient of a function of the downscaled lower-resolution image. An image is upscaled based at least in part on the regression model.