Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a neural network model to predict mesh vertices corresponding to a three-dimensional surface geometry of an object depicted in an image.