A method of video processing includes making a determination for a conversion between a video region of a video and a bitstream representation of the video to use a cross-component adaptive loop filtering (CC-ALF) tool for refining chroma samples values using luma sample values; and performing the conversion based on the determination, wherein the refining includes correcting the chroma sample values using a final refinement that is a further refinement of a first refinement value determined by selectively filtering the luma sample values.