A vehicular control system includes a camera having an exterior field of view at least forward of the vehicle. During a forward maneuver of the vehicle towing a trailer, the vehicular control system detects an object present exterior of the vehicle which ought not be impacted during the forward maneuver of the vehicle towing the trailer based at least in part on image processing by an image processor of image data captured by the camera. Responsive at least in part to detection of the object, the vehicular control system determines a forward driving path for the vehicle towing the trailer that avoids the detected object so that the trailer does not run over or contact the detected object. The vehicular control system determines the forward driving path at least in part responsive to (i) processing of captured image data and (ii) trailer data pertaining to physical characteristics of the trailer.