The decode time of big frames may be reduced by starting big frame decoding much earlier than normal decoding order. In a multi-threaded decoder, decoding one frame can be divided into several stages and stages of different frames can run in parallel like a pipeline. The first stage may be the token parser which is the most time consuming among all the stages because it is sensitive to bitstream size. If one or more stages of big frame decoding can be done much earlier than normal decoding order, the decode time spent exclusively on the big frame (not including the time shared with other frame decoding) may be much less in some embodiments. In this way, big frame decoding time may be reduced and in addition, depending on how big the big frame is, the decoder control can control how early to start decoding, so that frame drops caused by big frames can be reduced or eliminated.