The invention provides a video codec. In one embodiment, the video codec is coupled to an outer memory storing a reference frame, and comprises an interface circuit, an in-chip memory, a motion estimation circuit, and a controller. The interface circuit obtains in-chip data from the reference frame stored in the outer memory. The in-chip memory stores the in-chip data. The motion estimation circuit retrieves search window data from the in-chip data with a search window, and performs a motion estimation process on a current macroblock according to the search-window data. The controller shifts the location of the search window when the current macroblock is shifted, marks a macroblock shifted out from the search window as an empty macroblock, and controls the interface circuit to obtain an updated macroblock for replacing the empty macroblock in the in-chip memory from the reference frame stored in the outer memory.