A medical apparatus includes: an information acquisition section configured to acquire position information and visual line information in an endoscope inserted into a predetermined organ; an image acquisition section configured to acquire an image obtained by observing inside of the predetermined organ from a predetermined viewpoint; an alignment section configured to convert the position information and the visual line information to a coordinate system of a model of the predetermined organ; a position calculation section configured to calculate position information of an inner wall of the predetermined organ opposing in a visual line direction from the predetermined viewpoint; an index generation section configured to generate an index defined based on a predetermined feature; and an image generation section configured to generate an image in which the index and position information of the inner wall are associated, for the image acquired by the image acquisition section.