The present invention provides a method (100) and device (200) for allocating bits to video pictures in a video compression system. A sequence of video frames or pictures is received, and the number of bits used to compress a previously encoded frame and a signal-to-noise ratio for the immediately previously encoded video frame are determined. Rate distortion model parameters are updated. Then, first, second, and third deviations are determined. Finally, the bit allocation for the frame is determined based on the first number of bits used to compress a previously encoded frame, the first deviation, the second deviation, and the third deviation.