A video encoder and video decoder may determine a set of adaptive loop filters, from among a plurality of sets of adaptive loop filters, on a per-block basis. Each set of adaptive loop filters may include filters from a previous picture, filters signaled for the current picture, and/or pre-trained filter. By varying the set of adaptive loop filters on a per-block basis, the adaptive loop filters available for each block of video data may be more adapted to local statistics of the video data.