There is provided an image processing apparatus and an image processing method which are capable of reducing a processing amount of a cost calculation, the image processing apparatus including: a filter processing section configured to perform a filter process of applying an offset to a pixel of a decoded image that is decoded; and a control unit configured to control a mode used when the filter process is performed and an offset for the mode according to an encoding parameter used when an image is encoded.