Methods and systems for improved adaptive loop filters (ALFs) used in post-processing stage of in-loop coding or the prediction stage of video coding. To account for various shortcomings, techniques to improve coding gains and visual quality of ALFs are discussed. First, refinement of ALF coefficients for each block is allowed wherein different units (used for class calculation, e.g., 2×2 sub-blocks in GALF) located in different blocks with the same class index may have different filters. Second, ALF filters can be modified or weakened to without signaling ALF filter coefficients.