A system for generating a depth map for an object in a three-dimensional (3D) scene includes an image capture sensor and a processor. The image capture sensor is configured to capture a plurality of images of the object at a plurality of different focal planes. The processor is configured to calculate a plurality of variance values for a plurality of image locations for each image captured by the image capture sensor. The processor is also configured to determine a peak variance value for the plurality of image locations based on the calculated variance values associated with the same image location for each of the plurality of images of the object at the plurality of different focal planes. The processor is also configured to generate the depth map for the object based on the determined peak variance value for each image location and the plurality of different focal planes.