A method synthesizes virtual images from a sequence of texture images and a sequence of corresponding depth images, wherein each depth images stores depths d at pixel locations I(x, y). Each depth image, is preprocessed to produce a corresponding preprocessed depth image. A first reference image and a second reference image are from the sequence of texture images. Then, depth-based 3D warping, depth-based histogram matching, base plus assistant image blending, and depth-based in-painting are applied in order to synthesize a virtual image.