An image processing apparatus includes: a plurality of micro-lenses arranged in a two-dimensional pattern so that a subject light through an image forming optical system enter there; a plurality of light receiving elements disposed in a vicinity of a focal position at rear side of the micro-lenses to correspond to the plurality of micro-lenses respectively that receive the subject light through the micro-lenses; an image synthesizing unit that synthesizes an image on a focal plane that is different from a predetermined focal plane of the image forming optical system based upon outputs from the plurality of light receiving elements; and a processing unit that, based upon at least an objective image in a vicinity of the plurality of micro-lenses, and an auxiliary image outside the vicinity of the plurality of micro-lenses, which are synthesized by the image synthesizing unit, performs a process to enhance a resolution of the objective image.