The invention relates to oil painting stroke simulation using neural networks. Oil painting simulation techniques are disclosed which simulate painting brush strokes using a trained neural network. Insome examples, a method may include inferring a new height map of existing paint on a canvas after a new painting brush stroke is applied based on a bristle trajectory map that represents the new painting brush stroke and a height map of existing paint on the canvas prior to the application of the new painting brush stroke, and generating a rendering of the new painting brush stroke based on thenew height map of existing paint on the canvas after the new painting brush stroke is applied to the canvas and a color map.