An edge detecting method detects edge segments by searching all of the search lines forming an image from an end of the image in a direction perpendicular to the edges. If a line whose edge segment cannot be detected exists, a search is made in all of the search lines from the vicinity of the center of the image toward the end of the image, whereby edge segments are detected. A linear edge is determined from edge segments. A plurality of edge candidates are obtained from the edge segments for all of the search lines, and an optimum candidate is selected from among the edge candidates. Ruled lines are extracted from the source document in the image, and an optimum candidate is selected based on a comparison with a ruled line. As a result, an edge of the source document can be detected with high accuracy even if an image on a background side is unstable, or if materials of the background and the source document are similar.