An image processing apparatus and method may estimate binocular disparity maps of middle views from among a plurality of views through use of images of the plurality of views. The image processing apparatus may detect a moving object from the middle views based on the binocular disparity maps of the frames. Pixels in the middle views may be separated into dynamic pixels and static pixels through detection of the moving object. The image processing apparatus may apply bundle optimization and a local three-dimensional (3D) line model-based temporal optimization to the middle views so as to enhance binocular disparity values of the static pixels and dynamic pixels.