In automatic defect classification, a classification recipe must be set for each defect observation device. If a plurality of devices operate at the same stage, the classification class in the classification recipes must be the same. Problems have arisen whereby differences occur in the classification class in different devices when a new classification recipe is created. This defect classification system has a classification recipe storage unit; an information specification unit, the stage of a stored image, and device information. A corresponding defect specification unit specifies images of the same type of defect from images obtained from different image pickup devices at the same stage. An image conversion unit converts the images obtained from the different image pickup devices at the same stage into comparable similar images; and a recipe update unit records the classification classes in the classification recipes corresponding to the specified images of the same type of defect.