A digital signal decoder system is provided for receiving digital video signals and processing them while reducing the external memory requirements for frame buffer storage for an MPEG-2 decoder through decimation. The system includes a motion compensation unit for processing macroblock data. The decoder portion of the decimation unit intercepts and processes the data after it has been processed by the motion compensation unit. The data is then stored in the decimate buffer before passing on to the memory control unit. From here the data is routed to the display portion of the decimation unit for further processing. At this point the data is stored in the video buffer where it then passes on to the expansion filter and then to the display. As the video data is routed through the system, it could be decimated, interpolated, reduced, expanded, or any combination of these. The decoder controller controls and synchronizes the system operation. The final result is a system for decoding digital video signals while reducing the external memory requirements. This decoding system can be used to provide features that are desirable for MPEG-2 playback. These include display format conversion to letterbox, and display format conversion to 3/4 small picture size with additional storage allocation for on screen graphics.