An object recognition system and process that identifies people and objects depicted in an image of a scene. In general, this system and process entails first creating model histograms of the people and objects that it is desired to identify in the image. Then, the image is segmented to extract regions which likely correspond to the people and objects being identified. A histogram is computed for each of the extracted regions, and the degree of similarity between each extracted region histogram and each of the model histograms is assessed. The extracted regions having a histogram that exhibits a degree of similarity to one of the model histograms which exceeds a prescribed threshold is designated as corresponding to the person or object associated with that model histogram. Finally, the histogram computed for any extracted region of the image that is designated as corresponding to a person or object associated with a model histogram can be stored as an additional model histogram associated with that person or object. Preferably, the foregoing general system and process is repeated for subsequently generated images of the scene, so that the identity of people and objects can be monitored over time as they move into and about the scene. In addition, preferably color images of the scene and color histograms are employed in the object recognition system and process.