A magnifying observation apparatus obtains a plurality of first luminance images by controlling an illumination section so as to illuminate an observation target with illumination light from a first illumination direction and controlling a change section and an imaging section so as to image the observation target, obtains a plurality of second luminance images by controlling the illumination section so as to illuminate the observation target with illumination light from a second illumination direction symmetric with the first illumination direction about an optical axis and controlling the change section and the imaging section so as to image the observation target in a plurality of different focal positions, and generates a roughness enhancement image that enhances roughness on the surface of the observation target by applying depth synthesis and roughness enhancement.