Techniques and systems are provided for processing video data. For example, techniques and systems are provided for compensating for lighting changes in one or more video frames. To perform the lighting change compensation, a current frame and a background picture are obtained. A frame-level lighting condition change is then detected for the current frame. A block-level comparison of the current frame and the background picture is performed when the frame-level lighting condition change is detected. The block-level comparison includes comparing a block of pixels of the current frame with a corresponding block of pixels of the background picture. Based on the block-level comparison, it is determined that a change in the block of the current frame relative to a previous frame is associated with a change in lighting. Blob-level lighting compensation can also be performed.