Certain aspects and features of this disclosure relate to providing a hybrid approach for camera pose estimation using a deep learning-based image matcher and a match refinement procedure. The image matcher takes an image pair as an input and estimates coarse point-to-point feature matches between the two images. The coarse point-to-point feature matches can be filtered based on a stability threshold to produce high-stability point-to-point matches. A perspective-n-point (PnP) camera pose for each frame of video, including one or more added digital visual elements can be computed using the high-stability matches and video frames can be rendered, each using its computed camera pose.