A trailer angle detection system for a vehicle includes a camera disposed at a rear portion of the vehicle and viewing rearward of the vehicle. A plurality of ultrasonic sensors is disposed at the rear portion of the vehicle and sense rearward of the vehicle. A control has at least one processor operable to process image data captured by the camera. Responsive to processing of image data captured by the camera, the control detects a trailer rearward of the vehicle and in the field of view of the camera. The at least one processor is operable to process sensor data captured by the ultrasonic sensors to determine a distance to portions of the trailer rearward of the vehicle. Responsive to processing of captured image data and processing of captured sensor data, the trailer angle detection system is operable to determine an angle of the trailer relative to the vehicle.