Systems and methods for image-based bridge identification and boundary detection are provided. One example method includes determining, by one or more computing devices, a wide bridge mask and a narrow bridge mask based at least in part on a road mask and a water mask. The method includes selecting, by the one or more computing devices, a portion of an image depicting a bridge based at least in part on the wide bridge mask and the narrow bridge mask. The method includes identifying, by the one or more computing devices, a plurality of line features included in the portion of the image depicting the bridge. The method includes determining, by the one or more computing devices, a bridge boundary in the portion of the image based at least in part on the plurality of line features. One example system includes a bridge image extraction module and a bridge boundary determination module.