A vehicular vision system includes a rearward viewing camera and a control, which includes a processor that processes image data captured by the camera. During a reversing maneuver of the vehicle, the control, via processing of captured image data, detects points of interest of detected objects. Responsive to detection of the points of interest of the detected objects, the control determines if the detected objects have height dimensions and determines that objects having a height dimension greater than a threshold height are objects of interest. During the reversing maneuver of the vehicle, the control determines distances to the determined objects of interest and determines which object of interest is closest to the vehicle. During the reversing maneuver of the vehicle, responsive to the determination of the closest object of interest rearward of the vehicle, the control determines whether the determined closest object of interest is a potential hazard.