Stereoscopic video is encoded and decoded by using the MAC defined by the existing MPEG-4 standard. The stereoscopic video is divided into one image as a single video object, and another image as auxiliary information for the image established as the video object. The auxiliary information includes a horizontal disparity map, a vertical disparity map, luminance residual texture, and chrominance residual texture, which are respectively allocated to auxiliary components of the MAC according to importance and complexity of the images, encoded, and then output as a single encoding stream.