In some embodiments, macroblock-level encoding parameters are assigned to weighted linear combinations of corresponding content-category-level encoding parameters. For example, a macroblock quantization parameter (QP) modulation is set to a weighted linear combination of content category QP modulations. Content categories may identify potentially overlapping content types such as sky, water, grass, skin, and red content. The combination weights may be similarity measures describing macroblock similarities to content categories. A macroblock may be associated with multiple content categories, with different similarity levels for different content categories. A similarity measure for a given macroblock with respect to a content category may be defined as a number (between 0 and 8) of neighboring macroblocks that meet a similarity condition, provided the macroblock meets a qualification condition. The similarity condition may be computationally simpler than the qualification condition. Content-category-level statistics are generated by combining block-level statistics. Content-category-level statistics may be used in encoding subsequent frames.