The present invention discloses a multispectral stereo camera self-calibration algorithm based on track feature registration, and belongs to the field of image processing and computer vision. Optimal matching points are obtained by extracting and matching motion tracks of objects, and external parameters are corrected accordingly. Compared with an ordinary method, the present invention uses the tracks of moving objects as the features required for self-calibration. The advantage of using the tracks is good cross-modal robustness. In addition, direct matching of the tracks also saves the steps of extraction and matching the feature points, thereby achieving the advantages of simple operation and accurate results.