There is provided an image processing apparatus capable of easily removing or suppressing an area which need not be copied in an image. A characteristic amount calculation section detects rectangular areas physically/logically connected to each other from an image read by a scanner, and calculates unnecessary area information required to detect the characteristic amount and unnecessary area of the rectangular areas. An unnecessary area identifying section compares the characteristic amount of each object with unnecessary area identifying reference information based on the unnecessary area information to detect whether the object is an unnecessary area. An output processing section removes or filters the detected unnecessary area according to a specified method to store the unnecessary area in an output image storage section.