System and method for diagnosing a disease using image classification is provided. The method includes receiving diagnostic images associated with a subject. The diagnostic images comprise a first image and a second image captured at a first instance of time and at a second instance of time, respectively. The method further includes extracting feature maps from the first image and the second image, using a first pipeline and a second pipeline of stacked convolutional layers of a temporal convolutional neural network (CNN) model respectively. The method further includes identifying relevant feature maps from the feature maps using temporal CNN layer of the temporal CNN model, generating a concatenated feature map based on the relevant feature maps, determining an image class corresponding to the second image using fully connected layers of the temporal CNN model, and diagnosing the disease associated with the subject based on the image class.