An inspection apparatus and a method of inspecting a semiconductor device are disclosed. The inspection apparatus includes a stage on which a semiconductor device is positioned, a first light source irradiating a high-frequency light onto an inspection area of the semiconductor device to reduce a potential barrier of a PN junction in the semiconductor device, a beam scanner arranged over the semiconductor device and irradiating a charged particle beam onto the inspection area of the semiconductor device to generate secondary electrons, and a defect detector generating a detection image corresponding to the inspection area and detecting, based on a voltage contrast between a reference image and a plurality of detection images, a defect image indicating a defect in the semiconductor device from among the plurality of detection images.