The present invention discloses a method and a system for prostate multi-modal MR image classification based on a foveated residual network, the method comprising: replacing convolution kernels of a residual network using blur kernels in a foveation operator, thereby constructing a foveated residual network; training the foveated residual network using prostate multi-modal MR images having category labels, to obtain a trained foveated residual network; and classifying, using the foveated residual network, a prostate multi-modal MR image to be classified, so as to obtain a classification result. In the present invention, a foveation operator is designed based on human visual characteristics, blur kernels of the operator are extracted and used to replace convolution kernels in a residual network, thereby constructing a foveated deep learning network which can extract features that conform to the human visual characteristics, thereby improving the classification accuracy of prostate multi-modal MR images.