A restoration process using a restoration filter that is based on a point spread function for an optical system is performed for original image data acquired from an image-capturing element by an image taking of an object image using the optical system, so that recovery image data are acquired. The above restoration process is performed, in a point-image restoration processing section, for color data of the original image data in which a gradation correction has been performed by a logarithmic process. The restoration filter may be configured by a filter coefficient corresponding to the image data before the logarithmic process, or may be configured by a filter coefficient corresponding to the image data after the logarithmic process. Thus, by performing the restoration process capable of flexibly responding to various properties of the original image, it is possible to reduce the image degradation such as the ringing in the recovery image.