Implementations of the present disclosure provide a solution for reordering merge candidates. In this solution, during a conversion between a current video block of a video and a bitstream of the video, a template matching cost associated with each of a plurality of candidates of motion information for the current video block is determined. The template matching cost is determined based on a first difference between a set of neighboring samples of the current video block and a set of reference samples corresponding to the set of neighboring samples, and a second difference between a set of reconstructed samples neighboring to the set of neighboring samples and the set of reference samples. The set of reference samples is determined based on motion information indicated by a respective candidate. Then, the plurality of candidates of motion information is reordered based on the template matching cost.