An image processing device of the present invention acquires the positional information of a boundary line for a case where a first image is segmented into a foreground and a background by a segmentation processing that segments an image into a foreground and a background, and controls the segmentation processing to segment a second image having a predetermined relationship with the first image by use of the positional information of the acquired boundary line.