The invention discloses a bilateral video target segmentation method and system. According to the method, a video sequence with a key frame mark is mapped to a high-dimensional bilateral space, the video data to be processed is reduced, and then the non-empty grid unit is taken as the node of the graph and the graph cut optimization model is constructed; the method is characterized in that a confidence dynamic appearance model is constructed by analyzing a Gaussian distribution rule, and the possibility that each grid unit belongs to a foreground/background is accurately estimated; a high-order item is introduced into the energy function to enhance the space-time correlation of non-adjacent nodes with similar appearance characteristics; and finally, the energy function is solved by using amaximum flow/minimum cut algorithm to obtain a label value corresponding to each grid unit, and finally label distribution of the video pixel points is achieved. According to the method, interferenceof adverse factors on segmentation can be well eliminated, and video target segmentation with a complex scene can be rapidly processed with high precision.