A video decoding apparatus includes: a decoder to determine a size of a minimum block unit among block units in which quantization parameter information is decoded from a bitstream, determine a current block to be decoded by dividing a block larger than 16×16 into one or more blocks, decode, from the bitstream, a skip information and the motion information for deriving the motion vector of the current block without decoding information on residual signals of the current block, and decode, from the bitstream, prediction information of the current block and quantized transform coefficients of the current block; and a video reconstructor to determine the motion vector of the current block using the decoded motion information, reconstruct residual signals from the quantized transform coefficients by inversely quantizing the quantized transform coefficients using a quantization parameter, and reconstruct the current block based on the prediction information and the reconstructed residual signals.