According to the present disclosure, a video analysis method and a video analysis system in which video analysis accuracy at a cloud server and an edge is improved is provided. The video analysis method includes a first image analysis step of analyzing an input image frame on the edge side, a difference value estimation step of estimating a difference value between an evaluation value of an analysis result in the first image analysis step and an evaluation value of an analysis result being predicted when the input image frame is analyzed by the cloud server, and a filtering step of determining whether to transmit the input image frame to the cloud server, based on the difference value.