The invention discloses a refractory brick surface scratch identification method based on frequency domain filtering enhancement. The refractory brick surface scratch identification method comprises the steps of: acquiring a refractory brick grayscale image after segmentation; converting the refractory brick grayscale image into a refractory brick color image, performing channel decomposition on the refractory brick color image to separate out images of three channels R, G, and B, and acquiring a defect channel image; performing two-dimensional discrete Fourier transform on the defect channelimage, and then inversely transforming the transformed defect channel image to obtain a refractory brick image after inverse transformation; and performing threshold processing and morphological processing on the refractory brick image after inverse transformation to obtain a refractory brick threshold image, adopting a scanning method two times for the refractory brick threshold image, and marking all communicated regions existing in the refractory brick threshold image. The identification method for effectively identifying scratches on the surface of a refractory brick is provided by the invention.