The disclosure is directed to creating an inertial sensor aided depth map of a scene. An embodiment of the disclosure captures at least a first image and a second image during movement of a device caused by a user while framing or recording the scene, compensates for rotation between the first image and the second image, calculates an amount of translation of the device between the first image and the second image, calculates a pixel shift of a plurality of key points of the first image and the second image, and estimates a depth to one or more of the plurality of key points of the first image and the second image.