An artificial neural network system for recognizing a lesion in a fundus image, comprising: a pre-processing module, configured for pre-processing a target fundus image and a reference fundus image of the same person; a first neural network (12), configured to generate a first advanced feature set from the target fundus image; a second neural network (22), configured to generate a second advanced feature set from the reference fundus image; a feature combination module (13), configured to merge the first advanced feature set and the second advanced feature set to form a feature combination set; and a third neural network (14), configured to generate, according to the feature combination set, a judgment result of the lesion. The present invention uses a target fundus image and a reference fundus image as independent input information, and can thus stimulate a diagnostic process of a doctor to judge the target fundus image with reference to other fundus images of the same person, facilitating in improving the accuracy of judgment of the lesion from the fundus images.