A video coding system receives as input a video sequence including a series of picture frames. One or more long term references are selected from the input video sequence, at least one of the long term references is a long term look-behind reference frame. Short term reference frames are also selected according to the standards. The frames are then re-ordered for encoding such that the long term look-behind reference is encoded first, followed by the remaining frames according to the conventional order dictated by the standards. Each frame is encoded according to motion estimation and motion compensation, and an intra prediction method that incorporates the use of the long term look-behind reference frame. Further, encoding of each long term look-behind reference frame includes quantization according to a controlled bit-rate. The bit-rate is increased for quantization of each long term look-behind reference frame, thereby increasing its quality. For each other frame, the bit rate is maintained at a normalized level.