This disclosure is directed to video coding techniques that support normal single layer video coding, or scalable video coding with features such as signal-to-noise ratio (SNR) scalability and spatial scalability. A video coding device may implement these techniques in a video decoder that includes a motion compensation module and a filter. The motion compensation module decodes a prediction frame from a digital video signal, wherein the motion compensation module determines each block of the inter-coded frame from motion vectors encoded in the digital video signal. The filter adaptively filters one or more of the inter-coded blocks based on a signal either encoded or inferred from the digital video signal. In some instances, the video decoder may adaptively apply different filter functions, one in the horizontal and another in the vertical direction, based on the signal. By implementing these techniques, the video decoder may increase the visual quality of the resulting decoded digital video signal while reducing complexity.