The invention proposes a spatio-temporal data stream video behavior recognition method based on deep learning. Compared with a traditional deep learning method which utilizes a signal data stream, namely a single video stream, the spatio-temporal data stream video behavior recognition method based on deep learning utilizes spatio-temporal data streams, that is a spatial stream and a temporal stream, the temporal stream recognizes category of a target in a video from a static video frame, the spatial stream recognizes motion of the target from motion components in the video, and classification results of the temporal stream and the spatial stream are integrated to obtain a final behavior category. The spatio-temporal data stream video behavior recognition method based on deep learning recognizes the target and the motion separately, can reduce calculation burden of a neural network, and effectively increases accuracy.