An image classification system includes a convolutional neural network, a confidence predictor, and a fusion classifier. The convolutional neural network is configured to assign a plurality of probability values to each pixel of a first image of a scene and a second image of the scene. Each of the probability values corresponds to a different feature that the convolutional neural network is trained to identify. The confidence predictor is configured to assign a confidence value to each pixel of the first image and to each pixel of the second image. The confidence values correspond to a greatest of the probability values generated by the convolutional neural network for each pixel. The fusion classifier is configured to assign, to each pixel of the first image, a feature that corresponds to a higher of the confidence values assigned to the pixel of the first image and the second image.