For each prediction candidate of a set of one or more prediction candidates of the current block, a video coder computes a matching cost between a set of reference pixels of the prediction candidate in a reference picture and a set of neighboring pixels of a current block in a current picture. The video coder identifies a subset of the reference pictures as major reference pictures based on a distribution of the prediction candidates among the reference pictures of the current picture. A bounding block is defined for each major reference picture, the bounding block encompassing at least portions of multiple sets of reference pixels for multiple prediction candidates. The video coder assigns an index to each prediction candidate based on the computed matching cost of the set of prediction candidates. A selection of a prediction candidate is signaled by using the assigned index of the selected prediction candidate.