An apparatus for providing motion estimation for video encoding includes a selection element and a processing element. The selection element is configured to select a subset including less than all of candidate pixel locations from among a plurality of candidate pixel locations used for motion vector determination based on a relationship between a best candidate pixel location of a first level of accuracy and a best candidate pixel location of a second level of accuracy. The processing element is configured to process an input video sequence to determine a motion vector at the first level of accuracy, to refine the motion vector at the second level of accuracy, and to determine the motion vector at a third level of accuracy using only the subset of candidate pixel locations.