The present invention discloses a video-based target search and tracking method. The method comprises that: a service center receives a search request and video data sent by a terminal, wherein the search request comprises a search target and service requirements, and the service requirements comprise at least one of target search, target tracking, address recognition, route recommendation, and areal-time picture; the video data is input into a pre-trained deep learning network for target recognition to obtain a target recognition result; and according to the service requirements, service processing is performed on the target recognition result to obtain corresponding service information. According to the method disclosed by the present invention, by using the artificial intelligence theory, automatic search and tracking of the specified target can be realized in the video data through the deep learning method, and the corresponding service processing is performed on the target recognition result according to the service requirements of the user, so that the service information satisfying the user demand can be obtained. The present invention also discloses the service center, a terminal and a video-based target search and tracking system.