A method for medical image segmentation includes accessing and updating a knowledge-base. A medical image is received and a sparse landmark signature is computed based on the medical image. Either a representative or a cohort average reference image set is selected. A portion of either representative reference image set or the cohort average reference image set is deformed to generate mappings to the medical image set. A segmentation for each structure of interest of the medical image set is determined. The knowledge-base is searched for representative matches to form a plurality of sub-volume base sets comprising a plurality of reference image set sub-volumes. A portion of the plurality of reference image set sub-volumes is deformed to generate mappings from the plurality of sub-volume base sets to corresponding structures of interest of the medical image set. A weighted-average segmentation for the structures of interest in the medical image set is calculated.