A method and apparatus for tracking objects used in connection with athletic activities or sporting events, especially, balls, pucks, and the like. The method includes the steps of differencing present and previous frames of a video image including the, for example, ball to obtain motion regions, converting the motion regions to HSV color space, extracting the region corresponding to the ball based on empirical color data about the ball, obtaining a motion vector based on the motion of the ball region from a previous frame to the current frame, and updating the ball trajectory based on the newest motion vector obtained. The method also preferably includes a step of identifying completed trajectories based on preset constraints. The method is preferably expanded on by using at least one pair of cameras to provide a three-dimensional trajectory, and sometimes preferable expanded on by using a plurality of cameras, especially a plurality of pairs of cameras. An apparatus according to the present invention includes at least one camera connected to a computer which operates to difference previous and current frames, compute the ball track, convert ball regions to HSV color space and output the tracking and video information. In a case where one or more pairs of cameras are used, the computer is preferably also provided with a stereo matching device or module for matching the tracking results from respective cameras and/or respective pairs of cameras.