A vehicular control system includes a forward viewing camera and a non-vision sensor. The control, responsive to processing of captured non-vision sensor data and processing of frames of captured image data, at least in part controls the equipped vehicle to travel along a road. The control, responsive at least in part to processing of frames of captured image data, determines lane markers on the road and determines presence of another vehicle traveling along the traffic lane ahead of the equipped vehicle and determines a trajectory of the other vehicle relative to the equipped vehicle. The control stores a trajectory history of the determined trajectory of the other vehicle relative to the equipped vehicle as the other vehicle and the equipped vehicle travel along the traffic lane.