An image processing device appropriately extracts an entire object from an image even if the object is not completely included in an initial region designated by a user. The image processing device includes: a designation unit designating a position on the image; a setting unit setting a first combination consisting of a first object region candidate and a first background region candidate, and a second combination; a histogram generation unit generating a normalized color histogram of each of the regions; a similarity calculation unit calculating a similarity regarding each of the combinations; and a selection unit configured to select a combination from the first and second combinations so that the selected combination has the similarity lower than the similarity of the non-selected combination.