This application discloses a computing system to implement vehicle localization in an assisted or automated driving system. The computing system can receive an environmental model populated with measurement data captured by sensors mounted in a vehicle. The computing system can detect a location of the vehicle relative to the map data based on a correlation between the measurement data and the map data. The computing system can detect landmarks in the map data and switch to sparsely-populated map data from higher-definition map data for subsequent location detections. When the computing system does not detect a vehicle location, the computing system can track movement of the vehicle based on subsequent measurement data in the environmental model. After reacquiring a vehicle location, the computing can realign the tracked movement of the vehicle and measured data to the map data or modify the map data to include the tracked data.