A video encoder that utilizes adaptive interpolation filtering for coding video data includes a prediction unit, a reconstruction unit, a reference picture buffer, a filter parameter estimator for estimating filter parameters according to the original video data and the predicted samples, and an adaptive interpolation filter for utilizing the stored filter parameters to perform filtering on the reconstructed video data.