An image processing apparatus and an encoding device able to raise an encoding efficiency and a quality of a decoded image in comparison with a conventional apparatus and methods of the same, specifically image processing apparatus for processing a plurality of blocks defined in a two-dimensional image region in units of blocks, comprising a difference detecting unit for detecting the difference between a color difference enhancement block enhancing a color difference component with respect to a luminance component of a block to be processed in a picture to be processed or a block used for processing that block to be processed and the luminance block of the luminance component obtained from that block and a processing unit for performing processing strongly reflecting an influence of the color difference component of the block to be processed or processing not causing loss of information of the color difference component even when the difference detected by the difference detecting unit exceeds a predetermined threshold value compared with a case where the difference does not exceed the predetermined threshold value.