A surface inspection system detects imperfections on the surface of a hot radiant material, in which video signals for imperfections or defects are discriminated by comparison with normal level signals which are obtained through a variable area peak hold to preclude completely the influences of dark or bright defects which happen to be on the inspecting surface. The system includes apparatus for removing noises due to scales to produce outputs indicating true surface imperfections alone or indicating the degree of defectiveness of the detected defects. Also disclosed is a multi-camera surface inspection system capable of detecting defects on all sides of a hot radiant material which is transferred along a predetermined transfer line, with or without follow-up control of a camera or cameras scanning a given side of the material using edge signals detected from the normal level signal of another camera scanning a surface on the ensuing side of the material.