A technique is provided for programmably horizontally filtering pixel values of frames of a plurality of video frames. The technique includes, in one embodiment, passing pixel values through a real-time horizontal filter disposed as preprocessing logic of a video encode system. The horizontal filter is programmable and includes a filter coefficients buffer for holding multiple sets of filter coefficients. The horizontal filter programmably employs the multiple sets of filter coefficients to selectively perform spatial noise filtering, or spatial noise filtering and image scaling on the pixels. The filter coefficients are also programmable and may be changed dynamically and repeatedly, with changes being applied at frame boundaries. When performing image scaling, multiple sets of filter coefficients are employed.