A pulsed light high-speed polarization locking method of a continuous variable quantum key distribution system is disclosed. An integral type optical detector is used for converting energy of a single pulsed light into a peak voltage of an output electric pulse in real time so as to achieve real-time measurement of light pulse energy without conducting high-speed data sampling and simplify the data acquisition and processing. By utilizing the FPGA hardware, a conditional simulated annealing algorithm is quickly operated to search the target polarization state and achieve high-speed polarization locking under the pulsed light. In addition, the power change of a local oscillator is monitored in real time to strengthen the resistance of the system against local oscillator dithering attack. The present invention can effectively solve a problem of rapid fluctuations of polarization state of pulsed light caused by complex external environments.