In various example embodiments, a variational technique is provided for texturing a polygon mesh based on source images of a scene. The technique combines information from multiple source images to generate a texture map with patches that are the most similar to corresponding patches of the source images. The texture map may be the direct result of an optimization that jointly searches for the texture map and patch correspondence, which is initially unknown. The optimization may use a variational formulation that scores every possible texture map using an energy function that includes a matching term that quantifies how similar patches of the texture map are to the corresponding patches of the source images, and a regularization term that quantifies how complex the patch correspondence is.