Disclosed herein are systems, methods, and non-transitory computer-readable storage media for processing image information using a more computationally efficient version of a previously developed HSEG (Hierarchical Segmentation) algorithm. The HSEG algorithm includes iterations intertwining two forms of region growing to produce a set of hierarchically related image segmentations. The first form is a commonly used approach in which regions are grown by merging the most similar spatially adjacent regions. The second form is a unique approach in which regions are grown by also merging spatially non-adjacent regions that are at least as similar as the spatially adjacent regions merged by the first form in the same iteration. The improved HSEG algorithm limits the regions considered for non-adjacent region merging in the second form of operation to regions having at least a minimum number of pixels. This minimum is dynamically controlled to optimize processing efficiency and image segmentation quality.