A system and method, which may be an offline method, extracts relevant image features about listing items in a network-based publication system for enabling image similarity searching of such listing items. When a seller lists an item, an image of the item is uploaded and may be sent to a picture processing service, which generates several digests. The digests are compressed data structures each representing a particular image feature such as edge, color, texture, or words. These digests are then stored in a search database, where the digests can be used to retrieve listings by image similarity at scale. A similar process can be performed for a query listing for searching the search database for items similar to the query listing.