A multi-format video decoder includes a bitstream buffer, a system controller, a bitstream decoding unit, an intra mode decoding unit and a shared prediction module. The system controller selectively generates a first control signal or a second control signal according to a video bitstream. The bitstream decoding unit generates a decoding information signal according to the video bitstream when receiving the first control signal. The intra mode decoding unit generates an intra mode signal when receiving the second control signal. The shared prediction module performs an AC/DC prediction upon a current block of the video bitstream to generate a current first prediction result according to the decoding information signal and performs an intra prediction upon the current block to generate a current second prediction result according to the intra mode signal. The shared prediction module includes shared components being utilized in the AC/DC prediction and the intra prediction.