Motion field estimation is used to predict motion within video blocks. A current block of a current frame is identified as a projection of a first reference block of a first reference frame onto the current frame. The projection uses a first motion vector of the reference block with respect to a second reference frame. A temporal motion vector candidate is determined for the current block by projecting the current block onto a second reference frame. The temporal motion vector candidate identifies a second reference block. A motion vector selected from a motion vector candidate list is used to generate a prediction block using the selected motion vector. The current block is coded using the prediction block.