A system and method for recognizing and labeling anatomical structures in an image includes creating a list of objects such that one or more objects on the list appear before a target object and setting the image as a context for a first object on the list. The first object is detected and labeled by subtracting a background of the image. A local context is set for a next object on the list using the first object. The next object is detected and labeled by registration using the local context. Setting a local context and detecting and labeling the next object are repeated until the target object is detected and labeled. Labeling of the target object is refined using region growing.