A method of processing media data. The method includes fusing a plurality of intermediate filtering results of an adaptive loop filter (ALF) processing unit with a fusion mode to generate a final filtering result; and performing a conversion between a video including the ALF processing unit and a bitstream of the video based on the final filtering result. A corresponding video coding apparatus and non-transitory computer-readable recording medium are also disclosed.