A method, system, and computer-readable storage medium for characterizing and representing images. A plurality of feature descriptors for a plurality of images are received, where each feature descriptor encodes a respective feature in an image, and where each feature descriptor is transformed into a plurality of sub-descriptors in accordance with a specified transformation scheme. The feature descriptors correspond to an image feature descriptor space for the plurality of images, and the sub-descriptors correspond to a plurality of subspaces of the image feature descriptor space, where the plurality of subspaces span the image feature descriptor space. Each subspace of the plurality of subspaces is quantized using cluster analysis applied to the sub-descriptors, thereby determining a respective one or more feature primitives for each subspace, where the feature primitives are useable to characterize image features.