A video processing system configured with DMA latency compensation is disclosed. This compensation helps minimize or otherwise mitigate shortages of data to the display, thereby improving the quality of displayed video. A relatively small line buffer is used to stage data for video processing. Should an underflow of data occur (where the buffer reading process is ahead of the buffer writing process), data is read from the previous line buffer. This not only prevents shortages of data to the display, but also provides data that is more likely to be relevant to the actual scene being displayed (as compared to random data).