A method and apparatus for providing multi-panel rate control of real-time video digital encoders such as an MPEG or H series including the steps of bit allocation, rate control and adaptive quantization. Bit allocation assigns a target number of bits per group of pictures, and per picture of each type. Rate control adjusts the quantization parameter at the MB level to achieve that target number of bits per picture. Adaptive quantization further modulates the parameter per MB using local activity measure. Bit allocation and rate control can be implemented through a central control unit, or a central digital signal processor (DSP) while adaptive quantization can be implemented at the local panel.