In a video encoder, pixel values of a macro-block are processed to determine an activity measure indicative of the type of content in the macro-block. Several techniques are employed for determining the activity measure of a macro-block. In an embodiment, a default quantization scale for quantizing a macro-block is modified based on the activity measure of the macro-block. In another embodiment, the macro-block is classified into one of multiple classes based on its activity measure. The default quantization scale for quantizing the macro-block is modified based on the classification of the macro-block. In yet another embodiment, an encoding mode to be used for encoding a macro-block is also determined on the basis of the class of the macro-block. Several of the techniques exploit the fact that the human visual system (HVS) has different sensitivities in perceiving a (rendered) macro-block or video frame, depending on the type of macro-block content.