A method of analyzing a digital video clip captured by a camera to determine candidate frames for subsequent key frame selection including providing a camera motion sensor in the camera so that information is provided during image capture regarding camera motion including translation of the scene or camera, or scaling of the scene; forming a plurality of video segments based on the global motion estimate and labeling each segment in accordance with a predetermined series of camera motion classes; extracting key frame candidates from the labeled segments and computing a confidence score for each candidate by using rules corresponding to each camera motion class and a rule corresponding to object motion.