A controller for an autonomous vehicle receives audio signals from one or more microphones. The outputs of the microphones are pre-processed to enhance audio features that originated from vehicles. The outputs may also be processed to remove noise. The audio features are input to a machine learning model that classifies the source of the audio features. For example, features may be classified as originating from a vehicle. A direction to a source of the audio features is determined based on relative delays of the audio features in signals from multiple microphones. Where audio features are classified with an above-threshold confidence as originating from a vehicle, collision avoidance is performed with respect to the direction to the source of the audio features.