A vehicular trailer assist system includes a camera disposed at a rear portion of a vehicle and having a field of view exterior and at least rearward of the vehicle. At least a portion of a trailer hitched to the vehicle is within the field of view of the camera. A control includes electronic circuitry and associated software, with the electronic circuitry including a processor operable to process image data captured by the camera, with the image data captured by the camera being representative of at least the trailer hitched to the vehicle. The control, responsive to processing of image data captured by the camera, determines at least one deficiency in a hitch connection between the vehicle and the trailer. Responsive to determination of the at least one deficiency, the control alerts a driver of the vehicle of the determined deficiency.