A movement of a small picture block in a frame of video signals is provided by recursively applying a gradient method to a picture block in a present frame and a preceding frame to provide a series of sub-movements until said sub-movement decreases less than a predetermined value, and the resultant movement is obtained by the sum of the sub-movements. The present invention provides high accuracy in estimation of movement of a block in a picture as compared with a conventional gradient method, and is attractive in hardware implementation.