According to the data amount and the nature of a video stream and an audio stream included in a program stream which is to be decoded, an audio and video processing apparatus performs processing while allocating higher priority to one or the other of the video and audio streams. Without any increase in circuit complexity or clock speed, this apparatus can reproduce streams in at least two formats, in which the ratios of the amounts of encoded signal data and of the amounts of calculation required for decoding for the video and the audio are very different.