Techniques related to coding video using out of loop inter motion estimation are discussed. Such techniques include performing simultaneous motion estimation for multiple blocks using merge candidates such that at least one of the blocks has non-final merge candidates, finalizing the merge candidates for the at least one block, and resolving reference to any non-final merge candidates that became invalid in the finalized merge candidates for final motion estimation.