A method of determining a ground line from an input image. The method includes determining a plurality of ground line candidates from the image, determining a certain band a central-line of which is a boundary between a G region and other regions in a Ground Building Sky (GBS) map of the image, and determining the ground line of the image by selecting a ground line candidate, among the plurality of ground line candidates, having the greatest extent crossing the certain band.