A method and an apparatus for detecting an object in the image pickup field by template matching and tracking the detected object. An intruding object is detected from an image acquired by an image pickup device which takes the image of an image pickup field. A template image is formed from the detected object and is stored in a storing unit. A template matching is performed between a present input image of the image pickup field and the template image stored in the storing unit to detect the location of that part of the object which has a maximum degree of coincidence with the template image. An edge detection of the object is performed over a predetermined search area in the present input image which area includes the detected part of the object having a maximum degree of coincidence. Based on results of the edge detection, the detected part of the object is corrected and is determined as the present location of the object. A part of the present input image having the corrected location is a new template image with which the template image stored in the storing unit is updated. Instead of a single template image, a plurality of template images may be used for template matching to assure the tracking of the object with even higher accuracy and stability.