A video decoder includes a mode of operation for reverse playback that includes dividing a sequence of coded video data into groups of pictures (GOPs) and further subdividing the groups of pictures into a plurality of spans. Each span is processed in reverse coding order by decoding the first key frame and any reference frames that precede the currently processed span in coding order, selecting a plurality of frames from the currently processed span for decode and display and further selecting for decode but not display any reference frames from the currently processed span that were omitted by the first selection. The method accommodates control parameters that define the number of spans selected from each GOP and the number of frames selected from each span for decoded and display.