An autonomous driving system configured to perform an autonomous driving of a vehicle includes a lane change determination unit configured to determine a necessity of an operation intervention for a lane change of the vehicle by a driver of the vehicle during the autonomous driving, a lane change request unit configured to request the driver to perform the operation intervention for the lane change of the vehicle when the lane change determination unit determines that it is necessary to perform the operation intervention for the lane change of the vehicle by the driver, and a mode switching unit configured to switch a steering torque mode for a lane keep control in the autonomous driving from a normal steering torque mode to a weak steering torque mode when the driver is requested to perform the operation intervention for the lane change of the vehicle.