There is provided a moving image extracting apparatus including a movement detecting unit which detects movement of an imaging apparatus at the time when imaging a moving image based on the moving image imaged by the imaging apparatus, an object detecting unit which detects an object from the moving image, a salient object selecting unit which selects an object detected by the object detecting unit over a period of predetermined length or longer as a salient object within a segment in which movement of the imaging apparatus is detected by the movement detecting unit, and an extracting unit which extracts a segment including the salient object selected by the salient object selecting unit from the moving image.