Groups of video frames can be compressed using reversed ordering. For a group of a video sequence having a display order, a determination is made as to whether to arrange the group in a reversed display order before a prediction process of an encoder for the group. Responsive thereto, the group is arranged in the reversed display order as an input order to the prediction process. Otherwise, the group is provided to the prediction process in the display order as the input order. In performing the prediction process, the group, received in the input order, is arranged in a coding order using a group coding structure that defines at least one backward reference frame for inclusion within a set of candidate reference frames. Multiple backward reference frames may also be defined. The group is predicted in the coding order using the set of candidate reference frames.