An emergency vehicle detection system for a vehicle includes a plurality of camera assemblies having a camera and at least one microphone. At least one processor processes image data captured by the cameras, and processes outputs of the microphones. The at least one processor processes the outputs of the microphones to detect a sound source indicative of a siren emanating from an emergency vehicle exterior of the equipped vehicle, and to determine the direction of the detected sound source. Responsive to detection of the sound source and determination of the direction of the detected sound source, the at least one processor processes image data captured by at least one of the cameras that has its field of view encompassing the detected sound source to confirm that the detected sound source is a siren emanating from an emergency vehicle.