The invention discloses a system and method for establishing a disease library. The system comprises a data mining module, a text conversion module, an image information extraction module and a database; the data mining module is used for collecting clinical medical information of a target disease from a predetermined medical subsystem; the clinical medical information comprises clinical text information and/or clinical image information; the text conversion module is used for converting the clinical text information collected by the data mining module into text information of a predeterminedstructure and obtaining standardized text data; the text conversion module comprises a trained long-term and short-term memory type recurrent neural network; the image information extraction module isused for extracting predetermined information from the clinical image information and obtaining structured image data; the image information extraction module comprises a trained convolutional neuralnetwork; the database is used for obtaining and and storing the standardized text data and the structured image data. The system and method for establishing the disease library can automatically establish the disease library of full-cycle and multi-dimensional data, and therefore the data integrity and scientific research availability of an existing disease library are greatly improved.