A system for generating painterly pictures comprises a processor and a memory storing instructions that, when executed, cause the system to receive an input image; add a base layer of brush strokes based on a sampling of underlying pixel values in the input image, the base layer comprising coarse large brush strokes that are well-distributed in a uniform pattern; add a uniform layer of brush strokes iteratively based on a sampling of underlying pixel values in the input image and directionally based on a gradient of the input image; and add a detail layer of brush strokes based on a determined saliency map. The disclosure also includes similar methods and computer program products for generating painterly pictures.