A method of extracting an image of a moving object comprises the steps of extracting edge-enhanced input image from an input image taken by an image pick-up element moving in a plane perpendicular to an optical axis, recollecting a background edge-enhanced image from the input edge image, transfer in parallel horizontally and vertically the background edge-enhanced edge image so to correspond to an input edge-enhanced image forwarded successively; extracting an edge image of a moving object by extracting the background edge-enhanced image from the input edge-enhanced image, and renewing the background edge enhanced image based on weighted mean of the background edge enhanced image and the input edge image.