A method of decoding video data includes determining, by a video decoder, a neighboring block in a current frame is inter coded. The method includes, in response to determining the neighboring block is inter coded, determining, by the video decoder, a template for a current block in the current frame based on a partial reconstruction of the neighboring block. The method includes determining, by the video decoder, a reference block in a reference frame corresponding to the template for the current block and determining, by the video decoder, motion vector information for the current frame based on the reference block and the template. The method includes generating, by the video decoder, a predictive block for the current block of video data based on the motion vector information and decoding, by the video decoder, the current block of video data based on the predictive block.