An image processing apparatus includes: a three-dimensional model structuring section configured to generate, when an image pickup signal related to a region in a subject is inputted from an image pickup apparatus configured to pick up an image of an inside of the subject, three-dimensional data representing a shape of the region based on the image pickup signal; and an image generation section configured to perform, on the three-dimensional data generated by the three-dimensional model structuring section, processing of allowing visual recognition of a boundary region between a structured region that is a region, an image of which is picked up by the image pickup apparatus, and an unstructured region that is a region, an image of which is yet to be picked up by the image pickup apparatus, and generate a three-dimensional image.