A system and method for detecting objects and background in digital images using range information includes receiving the digital image representing a scene; identifying range information associated with the digital image and including distances of pixels in the scene from a known reference location; generating a cluster map based at least upon an analysis of the range information and the digital image, the cluster map grouping pixels of the digital image by their distances from a viewpoint; identifying objects in the digital image based at least upon an analysis of the cluster map and the digital image; and storing an indication of the identified objects in a processor-accessible memory system.