A charged particle beam device that detects a secondary charged particle beam generated by irradiation of a sample by a primary charged particle beam, includes: an image shift deflector that shifts an irradiation region for irradiation of the sample by the primary charged particle beam; a magnetic sector that separates the primary charged particle beam passing therein from the secondary charged particle beam from the sample using a magnetic field generated therein; a correction mechanism that is placed off of a trajectory of the primary charged particle beam but on a trajectory of the secondary charged particle beam inside the magnetic sector, the correction mechanism deflecting the secondary charged particle beam passing through; and a controller that controls the correction mechanism according to a defined relationship between a shift amount by the image shift deflector and a correction amount by the correction mechanism.