A vehicular driver assistance system includes a camera disposed at a vehicle and a control having an image processor that processes image data captured by the camera to determine lane markings demarcating a traffic lane in which the vehicle is traveling and to estimate a path of travel for the vehicle to maintain the vehicle in the traffic lane in which the vehicle is traveling in situations where the lane markings demarcating the traffic lane in which the vehicle is traveling are not readily determinable. The control estimates the path of travel for the vehicle at least responsive to vehicle information. The system updates the estimated path of travel ahead of the vehicle even when no lane markings are determined present on the road ahead of the vehicle. The control adjusts processing of captured image data responsive at least in part to a driving situation that the vehicle is in.