The invention discloses a traffic violation detection method based on video tracking and pattern recognition, and belongs to the technical fields of image recognition and pattern recognition. The method comprises the following steps: firstly, defining violation modes, namely defining traffic light stop lines, driving directions, and areas and directions of violation of course change; secondly, monitoring and controlling a camera to shoot video images of intersections; and thirdly, analyzing the video by an embedded system, performing detection and measurement by a pattern recognition method, using the video to track and acquire running tracks of vehicles, using the pattern recognition method to analyze behaviors of any one vehicle according to the running tracks, and immediately shooting the vehicle for recording if one of the following violation behaviors happens: going in wrong directions, violation of full line course change and running of a red light. A detection device used comprises a high-speed camera, a video controller, and a video tracking and pattern recognition device, wherein the video tracking and pattern recognition device comprises a traffic light signal receiving device, a video acquisition and conversion module, a violation setting module, a vehicle detection module, a video tracking module, a violation analysis module, a camera control module and a data wireless transmission module.