An image processing apparatus may include a sensor unit which generates both a color image and a depth image. A color-depth time difference correcting unit may correct a mismatching that occurs due to a time difference between a point in time when the color image is generated and a point in time when the depth image is generated. An image generator may generate a stereoscopic image or a multi-view image using the color image and the depth image after the mismatching is corrected.