The present application discloses a method and device of detecting fatigue driving, comprising: analyzing an eye image in the driver's eye image area with a rectangular feature template to obtain the upper eyelid line; determining the eye closure state according to the curvature or curvature feature value of the upper eyelid line; and collecting statistics on the eye closure state and thereby determining whether the driver is in a fatigue state. The present application determines whether the eyes are opened or closed according to the shape of the upper eyelid, which is more accurate because the upper eyelid line has characteristics of higher relative contrast, anti-interference capacity, and adaptability to the changes in the facial expression.