Embodiments are directed towards modifying a quality of an image without affecting a bit rate associated with a video sequence of images. For each macro-block (MB) or sub-block within a MB for a target image to be encoded, various statistics are determined for a luminance component and chrominance components that provide at least average values for the chrominance components that may then be used to identify saturation thresholds. The saturation thresholds are used to determine a Qlevel. An average Qlevel from at least a previous frame is used to generate a Q modulation factor that may be combined with an activity based modulation factor and/or variance based modulation factor, or used singly. The final quantizer is calculated by bit-rate controller base quantizer multiplied by the Q modulation factor, and may be used to encode a MB or sub-block within a MB.