The invention discloses an angle-based wandering track detection method in video surveillance, which comprises the following steps of: 1, performing moving target detection on each acquired frame of image; 2, after a target is detected, determining positions A and C of the moving target in two initial frames, and connecting the target positions n the two initial frames to generate a radial AC; 3, similarly, taking the initial point A of the moving target in the initial frame as a starting point, and calculating an included angle alpha between a radial AD passing each target position D and the radial AC; 4, determining the type of a track according to the changing regularity of the included angle alpha, judging whether the track is a wandering track or not, and setting a wandering threshold with an initial value 0; and 5, judging whether the wandering threshold exceeds the set threshold or not, giving a wandering alarm if the wandering threshold exceeds the set threshold, otherwise returning to the step 1 to reacquire a video sequence. The method ensures high accuracy and high applicability.