A method of reducing false positives and identifying relevant true alerts in a video management system includes analyzing images to look for patterns indicating changes between subsequent images. When a pattern indicating changes between subsequent images is found, the video management system solicits from a user an indication of whether the pattern belongs to one of two or more predefined categories. The patterns indicating changes between subsequent images are saved for subsequent use. Subsequent images received from the video camera are analyzed to look for patterns indicating changes between subsequent images. When a pattern indicating changes between subsequent images is detected by the video management system, the video management system compares the pattern indicating changes between subsequent images to those previously categorized into one of the two or more predefined categories. Based on the comparison, the video management system may provide an alert to the user.