There is provided an image processing apparatus including a stereo matching unit configured to obtain right and left disparity images by using stereo matching, based on a pair of images captured by right and left cameras, respectively, a filter processing unit configured to perform filter processing on the disparity images, and a first merging unit configured to make a comparison, in the disparity images that have undergone the filter processing, between disparity values at mutually corresponding positions in the right and left disparity images and to merge the disparity values of the right and left disparity images based on a comparison result.