Binarization is performed using a threshold image obtained by multiplying a variation in each pixel value of an input image with a coefficient. Although the variation is time-averaged based on an update coefficient for each pixel, the update coefficient is switched depending on whether or not a relevant pixel belongs to the object. Subsequently, from the binary image, an initial detection zone is formed and a spatial filtering process is performed thereto. The spatial filtering process includes at least one of skeleton analysis processing, object mask processing, morphology processing, and section analysis processing. For a tracking zone, the temporal positional change thereof is tracked, and the noise is reduced. Some of the tracking zones are removed, and the remaining zones are integrated into a cluster, and furthermore the cluster selection is performed based on the dimensions in real space.