The endoscope insertion direction detecting method is to extract the form of a fold existing on the inside wall of an observed part from an endoscope picture image and to judge the direction of inserting the endoscope on the basis of the form of this fold and comprises, for example, a step of extracting discontinuous points in the endoscope picture image, a step of extracting line segments based on the above mentioned discontinuous points from respective divided picture images obtained by dividing into a plurality of picture images the picture image obtained by the step of extracting the discontinuous points, a step of connecting the line segments obtained by the step of extracting the line segments and a step of considering the line connected by the connecting step to be the form of a fold existing on the inside wall of the observed part and judging the endoscope insertion direction on the basis of this form of the fold. The step of extracting the discontinuous points includes, for example, extracting points at which the brightness and color of the picture image varies. The step of extracting the line segments has, for example, a step of extracting line segment candidates and a step of extracting the optimum line segment from among the candidates extracted by this candidate extracting step.