Systems and methods are described for estimating a camera pose. The estimation may include obtaining a sequence of images including a plurality of image frames of a scene, detecting a first set of feature points in a first image frame, and tracking the first set of feature points in a plurality of subsequent image frames. While continuing to track the first set of feature points, the estimation can include detecting a second set of feature points in a second image frame, tracking the second set of feature points, selecting a first initial camera pose associated with the first image frame and a second initial camera pose associated with the second image frame, determining projection locations, the projection locations based on the first initial camera pose and the second initial camera pose, and comparing the projection locations corresponding to each feature point in the first and second sets of feature points.