The invention provides a multi-instance learning idea-based training sample selection method during the target tracking process and relates to the target tracking technology in the field of computer vision. According to the technical scheme of the invention, the problem that poorly marked samples cannot be accurately marked during the target tracking process due to the conventional supervised learning algorithm in the prior art can be solved. According to the method, all positive samples are put into a positive sample packet. Samples having a smaller contribute to a log likelihood function with respect to the packet are considered to be poor samples. In the iteration mode, a worst sample is removed out of the packet during each iteration process till a sufficient number of samples are remained in the positive sample packet. Therefore, the method is superior to other assessment algorithms in the aspect of tracking accuracy. Meanwhile, the method is higher in frame rate and satisfies real-time requirements.