With intelligent differential quantization, a video codec intelligently quantizes video at differing strength levels within a frame, such as on a macroblock (MB) or a group of MB basis. This allows the codec to control bit usage on a finer granularity than a frame to meet hardware constraints, as well as providing perceptual optimization by coarsely quantizing unimportant regions, while finely quantizing important regions within a frame. The intelligent differential quantization uses motion information gathered from encoding and analysis of the video to classify the importance of different regions of the image, and quantizes the regions accordingly. In addition, the intelligent differential quantization include efficient signaling of information as to the differential quantization strengths in the compressed bit stream.