The invention discloses a classification method based on the depth learning characteristics of a longitudinal three-dimensional image, belonging to the field of image analysis. The invention comprisestwo stages: a training stage, which comprises a pretreatment of a three-dimensional image of a research object, a slice image feature extractor, a scanning image feature vector calculated according to a bag model, and a circulating neural network classifier designed and trained; a testing stage: putting forward the classification request to the object P to be classified, preprocessing the three-dimensional image of P, extracting the slice image feature, calculating the scanning image feature, and outputting the classification result. The invention utilizes depth convolution neural network toextract feature directly from slice image, the invention does not need any artificially designed image features, 3D registration of the image, feature region detection and other complex preprocessingsteps, and the invention studies the longitudinal three-dimensional image, learns the change features of the image from the longitudinal three-dimensional image feature vectors by using a circulatingneural network, and classifies the images more accurately.