A method for adaptively selecting feature points according to motions of objects between the current frame and the reference frame, comprising: detecting a motion vector for each pixel in the current frame between the current frame and the reference frame; providing a pixel value from the reference frame based on the motion vector for each pixel in the current frame; subtracting the pixel value provided from the reference frame from a pixel value of said each pixel of the current frame to thereby provide a differential pixel value; comparing the differential pixel value with a threshold value TH and selecting regions having their respective differential pixel values larger than the threshold value TH; shifting the pixels within the selected regions to positions indicated by their respective motion vectors to thereby provide shifted regions; detecting edge points from the reference frame; determining processing regions from the shifted regions, wherein the shifted regions are overlapped with a portion of the edge points; generating a first grid on the reference frame and a second grid for each of the processing regions, wherein the second grid is formed by a portion of grid points of the first grid and newly added grid points; and selecting, as the feature points, a multiplicity of pixels in the reference frame based on the first and the second grids and the edge points.