A segmentation of the esophagus from image data by specifying only the two end points is disclosed. Surrounding structures are used as high-level constraints to construct shape and appearance models. Prior shape information is integrated for the segmentation of a new esophagus using a Bayesian formulation. This permits to automatically select the proper models. Given the end points, a shortest path algorithm provides the optimal esophagus according to the Bayesian formulation.