A method and apparatus are disclosed and described for providing hypothetical reference decoder conformance error detection. The apparatus includes a multi-pass encoder (200) for encoding pictures in a video sequence. The pictures are encoded in a first pass to determine a bit consumption of each picture, and a bit allocation is controlled during the encoding of the pictures in at least a second pass responsive to satisfying requirements for a subsequent decoding of the bitstream. The requirements relate to preventing at least one of underflow and overflow conditions in a buffer during the subsequent decoding. The bit allocation is based on the bit consumption determined in the first pass and buffer parameters. The pictures are encoded in at least the first and second passes into a plurality of bitstreams on a scene-basis such that any of the pictures belonging to a same scene are respectively encoded in a same one of the bitstreams.