Example implementations include a method and of video processing, comprising determining, during a conversion between a current video unit of a video and a bitstream of the video, to apply a prediction mode derivation tool to the current video unit, wherein in the prediction mode derivation tool, at least one intra prediction mode (IPM) is derived for the current video unit based on cost calculations between reconstructed samples and prediction samples of a template region related the current video unit. The implementations further include performing the conversion based on the determining.