A method of detecting an anomaly is provided, including dividing each frame of a video stream into a plurality of cells; in each cell formulate statistics based on metadata generated for the frame, the metadata related to presence of an object in the cell, velocity of objects in the cell, direction of motion of objects in the cell, and classification of objects in the cell; and using the formulated statistics to determine when the anomalous activity has occurred in one of the cells of the plurality of cells.