A video coding system for coding a sequence of video frames each having an array of pixel values is provided. The system includes an interpolation filter to reconstruct sub-pixel values for use in the inter-frame prediction process. The coefficients of an interpolation filter are obtained differentially with respect to a predefined base filter in order to provide a set of difference values. As the base filter coefficients are known to both encoder and decoder and they are statistically close to the actual filters used in the video sequence, the decoder can reconstruct the pixel values based on the set of difference values.