Provided are an image processing apparatus and method. The image processing apparatus includes a decoder configured to decode image frames of an image; an image quality controller configured to obtain a genre recognition confidence of a previous image frame and a genre recognition confidence of a current image frame, and identify image quality control factor value, based on the genre recognition confidence of the previous image frame and the genre recognition confidence of the current image frame; and an image quality processor configured to image-quality process at least one of the decoded image frames by using the image quality control factor value, and output the processed at least one of the decoded image frames.