Provided is a sample observation apparatus including a charged particle optical column that irradiates a sample including an observation target portion that is a concave portion with a charged particle beam at an acceleration voltage, an image generation section that acquires an image including the observation target portion from a signal acquired with irradiation of the charged particle beam, a storage section that stores information representing a relationship between a brightness ratio of a concave portion and its neighboring portion of a reference sample that is irradiated with the charged particle beam at the acceleration voltage and a value that represents a structure of the concave portions of the reference sample in advance, a calculation section that acquires a brightness ratio of the concave portion and its neighboring portion of the image, and a determination section that determines whether or not a defect occurs in the observation target portion based on the information that represents the relationship and the brightness ratio of the image.