A method for modifying the viewpoint of a main image of a scene captured from a first viewpoint. The method uses one or more complementary images of the scene captured from viewpoints that are different from the first viewpoint. A warped main image is determined corresponding to a target viewpoint by warping the main image responsive to a corresponding range map, wherein the warped main image includes one or more holes corresponding to scene content that was occluded. Warped complementary images are similarly determined by warping the complementary images to the target viewpoint responsive to corresponding range maps. Pixel values to fill the one or more holes in the warped main image are determined using pixel values at corresponding pixel locations in the warped complementary images.