A motion estimation method and a motion estimator are provided. The motion estimation method includes the following steps of for each respective block of the current frame, selecting a candidate set associated with the current frame and a previous frame, and determining a motion vector according to the candidate set for the each respective block of the current frame. The candidate set includes at least one spatial candidate block in the current frame and temporal candidate blocks in the previous frame. The step of selecting the candidate set includes: selecting the spatial candidate block directly adjacent to the each respective block from the current frame and selecting the plurality of temporal candidate blocks directly adjacent to a reference block from the previous frame.