A perception and map edge association system for an autonomous vehicle includes one or more controllers executing instructions to receive a plurality of perception lane edge candidates that are each representative of a perception lane edge and a plurality of map lane edge candidates that are each representative of a map lane edge. The one or more controllers identify a discrepancy between a selected perception lane edge candidate and a best fit map edge candidate for a lane characteristic based on a posterior association probability matrix and a discrepancy flag. The one or more controllers segment the pair of inconsistent perception and map lane edges into one or more matching segments. The one or more matching segments indicate where the perception lane edge and the map lane edge match one another based on the lane characteristics. The perception and map edge association system improves localization and scene creation performance.