In a transmission side, a layer adjusting unit performs an adjustment to store a scalable encoded video data in a memory, select and output a scalable data of a base quality as a real-time video, read from a memory a scalable data of an enhancement quality based on the designation of a enhancement quality and a transmission period of time, and output the scalable data of an enhancement quality together with a real-time scalable data. In the receiving side, a layer synthesizing unit synthesizes and outputs a real-time video at normal times, and lowers the real-time video to a quality of the base layer if there is an enhancement quality video of a designated period of time, thereby reading the base quality layer data corresponding to the designated video from the memory (153) and synthesizing the data with the enhancement quality video to output the resulting data.