The invention provides a method and device for adjusting a target scale in a pan-tilt-zoom (PTZ) tracking process. The device comprises a bi-directional light stream processing module, a correlation coefficient matching module, an image scale generating module, an image segmentation module and a target scale fine tuning module. The method based on bi-directional light stream is provided to solve the problem of target size inaccuracy caused by zooming operation in the PTZ active tracking process, the accuracy rate of tracked feature point pairs is high, tracked feature points are correct, the feature point tracking accuracy rate and stability are greatly improved, and accordingly the stable and accurate target scale is obtained. The target scale reduces background interference and influence as far as possible, and the target location and scale estimation accuracy rate is further improved.