A video recognition system detects the presence of fire based on video data provided by one or more video detectors. The video recognition system is operable to calculate a first flicker feature with respect to a first set of frame data and a second flicker feature with respect to a second set of frame data. The video recognition system combines the first flicker feature and the second flicker feature to generate an accumulated flicker feature. The video recognition system defines, based on the accumulated flicker feature, a flicker mask that represents a dynamic region of the fire. Based on the defined flicker mask, the video recognition system determines whether the video data indicates the presence of fire.