The invention provides a video data processing method. The method includes the following steps: firstly receiving all data packets of each video frame, then detecting the packet loss of each video frame, based on the level of the packet loss, determining whether to add a first label to the video frame, if all video frames require the first label, then outputting video frames based on order, if there is a video frame which is not added with the first label, determining whether to add the first label to the video frame, based on the determination result, determining whether to re-transmit a command. According to the invention, the method, by conducting determination twice on the data packets of the video frames, and based on the level of packet loss, determines whether to re-transmit data packet, so that the method herein ensures the stability, smoothness and completeness of video outputting, and can save memory and bandwidth from being overly occupied by transmitting unnecessary sub-stream files.