A plurality of original images of a subject viewed from different viewpoints are acquired. Then, at least one interpolation image to interpolate at least one new viewpoint between the viewpoints of the original images or outside the viewpoints of the original images is generated from the original images. Then, identification information indicating whether each image is the original image or the interpolation image and viewpoint information representing the order of viewpoint of each image are added to the original images and the interpolation image, and the original images and the interpolation image with the identification information and the viewpoint information added thereto are stored in an associated fashion.