The invention discloses a rotation digit identification method and system based on a setpoint; the method comprises the following steps: 1, preprocessing an input video frame image so as to obtain a corresponding grey scale image; 2, extracting the grey scale image contour so as to obtain setpoint position, width and length of a color block in the grey scale image; 3, rotating and rectifying the grey scale image according to the setpoint position, thus obtaining the rectified image and rectified setpoint position; 4, segmenting the rectified image according to the rectified setpoint position, the color block width and length, thus obtaining a digit image only containing a digital portion; 5, graying the digit image, and using a depth learning model and a classifier to obtain digits in the digit image. The advantages are that the method and system can ensure the accuracy of the setpoint and rectification angle; the method is high in identification rate, simple in parameter setting, fast in training speed, and can be directly loaded into a trained network in real applications.