An adaptive temporal motion filter for a video decoder system operates in an infinite impulse response (IIR), a max or a bypass mode. The adaptive temporal motion filter includes an adaptive time constant control module and a filter gain module. A gain factor of the filter gain module is varied by the adaptive time constant control module for every pixel in a current composite video signal. The adaptive time constant control module selects a variable gain for the filter gain module based on the motion magnitude, motion polarity and chroma luma status of the pixel.