The invention provides a signal processing method of a semiconductor detector and a device. The method comprises a step of obtaining the relation between the time difference of the semiconductor detector anode and cathode signals and an anode signal amplitude, a step of obtaining an optimal data screening interval according to the relation between the time difference of the semiconductor detector anode and cathode signals and the anode signal amplitude, wherein the optimal data screening interval is an interval whose time difference of the anode and cathode signals is larger than 50ns, and a step of screening and processing the collected data according to the optimal data screening interval in the semiconductor detector data collection. According to the signal processing method and the device, the inherent defects of a detector crystal are overcome, the influence of ground noise is reduced, the energy resolution of a CZT detector is improved further, and the ratio of peak health is improved.