Systems for detecting movement of a trailer having a rearward facing camera and linked to a tow vehicle. The systems include taking a first image and a second image with the camera and comparing the images. A first amount of trailer movement is determined between the second image and the first image. The systems may also take additional images and determine a second amount of trailer movement occurring between the additional images. Tracking the first amount and the second amount of trailer movement determines an amount of trailer sway. The determined trailer sway may be compared to a maximum allowable sway and, if the movement is greater than the maximum allowable sway, mitigating sway of the trailer with the tow vehicle.