Systems and methods of signaling different filter structures for each pixel or sub-pixel position in motion compensation prediction video coding are provided. An encoder signals to a decoder one filter structure among a plurality of pre-defined candidates that is used for a respective pixel or sub-pixel position. In accordance with one embodiment, filter structures signaled to the decoder from the encoder “switch” between directional filter and radial filter structures during interpolation at the sub-pixel level. In accordance with another embodiment, filter structures that are signaled may switch between a directional filter structure and a separable filter structure at the sub-pixel level. Thus, not only can an encoder switch between different filter structures during interpolation, but a filter structure pair is provided that the encoder can utilize to interpolate a wide range of signals without increasing tap-length.