An apparatus that detects an obstacle using images shot by an image shooting device mounted on a mobile object on a plane, the apparatus comprising: an image receiving unit configured to receive time-series images shot by the image shooting device; an area setting unit configured to set a plurality of processing areas in the images received by the image receiving unit; an object motion detector configured to detect objects moving within the areas set by the area setting unit and to detect motion trajectories of the objects; an obstacle candidate detector configured to detect the respective objects as obstacle candidates if a direction of a line connecting between the objects is a predetermined direction; and an obstacle determining unit configured to compare the motion trajectories of the respective obstacle candidates and to determine the obstacle candidates as an obstacle if the motion trajectories of the obstacle candidates satisfy a predetermined similarity condition.