A method of rate control in coding of a video sequence to generate a compressed bit stream is provided that includes computing a sequence base quantization step size for a sequence of pictures in the video sequence, computing a picture base quantization step size for a picture in the sequence of pictures based on the sequence base quantization step size, a type of the picture, and a level of the picture in a rate control hierarchy, and coding the picture using the picture base quantization step size to generate a portion of the compressed bit stream.