This disclosure describes techniques associated with filtering of video data in a video encoding and/or decoding process. In accordance with this disclosure, filtering is applied at an encoder, and filter information is encoded in the bitstream to identify the filtering that was applied at the encoder. Different types of filtering may be applied based on an activity metric determined for the video data. Moreover, in accordance with this disclosure, the manner in which the filter information is encoded into the bitstream may be dependent on the activity metric. In particular, for a first range of the activity metric, one or more filters are encoded directly, and for a second range of the activity metric, one or more filters are predictively encoded.