Disclosed is an image encoding apparatus and a memory access method. The image encoding apparatus for encoding image data includes: a memory for storing a frame of the image data; a motion prediction unit for detecting a motion vector of a predetermined block unit for a currently-input image frame based on the stored frame, and accessing the memory in order to perform sub-pixel interpolation for a corresponding block of the motion vector; and a motion compensation unit for performing motion compensation with respect to a frame block to be compressed, by using the detected motion vector, thereby reducing the number of times the memory is accessed for encoding. Since the number of memory access times is reduced, which is very frequently used in interpolation of a color difference signal, low-power encoding is achieved.