The present invention discloses a system and method for adaptive selection of scanning modes based on the content of the input image sequence. In one embodiment, two encoders are employed. A first encoder receives the input image sequence and encodes each frame of the image sequence using at least two different scanning modes, e.g., zigzag scanning mode or alternative scanning mode in accordance with the MPEG-2 standard or the like. Specifically, different portions of each frame will be scanned using different scanning modes. This first encoding provides look-ahead information so that a second encoder is able to assign DCT quantized coefficients in a more efficient scanning order, thereby reducing encoding bits and/or improving the picture quality.