A non-keyframe reconstruction technique is described for selecting and reconstructing keyframes that have not yet been included in a reconstruction of an input image sequence to provide a better reconstruction in a structure from motion (SFM) technique. The technique may, for example, be used in an adaptive reconstruction algorithm implemented by a general SFM technique. This technique may add and reconstruct non-keyframes to a set of keyframes already generated by an initialization technique and reconstructed by adaptive and optimization techniques for iteratively selecting and reconstructing additional keyframes. Camera motion and intrinsic parameters may be computed for non-keyframes by optimizing a cost function. Output of the non-keyframe reconstruction technique may include at least camera intrinsic parameters and Euclidean motion parameters for the images in the input image sequence.