A video-based fire detection system receives video data comprised of a plurality of individual frames, and determines based on the video data the ability of the system to detect the presence of fire. The system includes a video recognition system connectable to receive the video data and to calculate one or more background features associated with the video data. Based on the calculated background features, the video recognitions system assesses the ability of the video-based fire detection system to detect the presence of fire. The system includes one or more outputs operably connectable to communicate the results of the assessment made by the video recognitions system.