Dominant color names may be extracted from an image by analyzing spatial-context of pixels contained in the image. A dominant color region may be defined by taking a double-threshold approach that addresses ambiguous color regions and a degree of confidence that each pixel belongs in the dominant color region. Affiliation maps and binary maps may be used to generate the dominant color region. Images may be converted to a saliency map, from which a region of interest may be assigned a dominant color name. Image search results may be filtered by the dominant color name associated with the image.