Techniques are described for performing multi-stage image classification. For example, multi-stage image classification can comprise a first classification stage and a second classification stage. The first classification stage can determine an overall classification for an input image (e.g., based on a relative entropy result calculated for the input image). The second classification stage can be performed by dividing the image into a plurality of blocks and classifying individual blocks, or groups of blocks, based on a classification model that is specific to the overall classification of the image determined in the first classification stage.