Picture data is compressed and encoded in consideration of visual characteristics of human beings against colors. A color of pixels is detected corresponding to color difference components Cb and Cr for each sub block into which a macro block is divided. Each sub block is composed of four pixels×four lines. The number of pixels from which a predetermined color is detected is counted for each sub block. When the count value is ⅔ or greater of the total number of pixels of each sub block, it is determined that the macro block which contains the sub block contains the predetermined color. An activity is obtained using a luminance component Y for the macro block. The activity is multiplied by a quantizer priority value corresponding to the predetermined color. As a result, the activity is corrected. Corresponding to the corrected activity, the quantizer scale value is changed to a smaller value. With the changed quantizer scale value, the macro block is quantized with a smaller quantizer step value. As a result, the picture quality of a macro block from which a predetermined color has been detected is suppressed from being deteriorated.