There is provided a video encoding apparatus allowing for enhanced video encoding speed according to the H.264 video coding standard. The video encoding apparatus allows the memories included in the video encoding apparatus to be shared by a plurality of elements through the rearrangement and the structural change of the memories considering an efficient hierarchical motion estimation algorithm. Therefore, the video encoding apparatus has the effects of reducing the amount of transmitted and received data between the frame memory and the video encoding apparatus and enhancing video encoding speed.