A method of adaptive loop filtering for reconstructed video to improve the performance is disclosed. In one embodiment, a loop filter is selected from one or more candidate filters including a circular symmetrical filter candidate. The candidate filters also may include a ½-symmetry filter candidate and a flag is used to select between different symmetries. In another embodiment, a priority-based block filter merge is disclosed. A filter merge list consisting of no more than a first number of filter candidates selected from corresponding filters used by corresponding set members in the neighboring block set is generated according to a priority order of the set members in the neighboring block set. In yet another embodiment, the loop filter is shared by reconstructed video in multiple image units of a same group. The filter merge groups are identified by differences of consecutive group indices.