A method and computer readable medium storing a corresponding computer program for decoding a video bitstream based on processors using dynamic load balancing are disclosed. In one embodiment of the present invention, the method configures multiple processors to perform the multiple processing modules including a prediction module by mapping the multiple processing modules to the multiple processors. One or more buffer queues are used among said multiple processing modules and the mapping the prediction module to the multiple processors is based on the level of the buffer queue. The multiple processors may correspond to a multi-core Central Processing Unit (CPU) comprising of multiple CPUs or a multi-core Digital Signal Processor (DSP) comprising of multiple DSPs to practice the present invention.