A hydraulic control system is disclosed for use in a machine. The hydraulic control system may have a work tool, a motor configured to swing the work tool, a tank, a pump configured to draw fluid from the tank and pressurize the fluid, and a control valve operable to control fluid flow from the pump to the motor and from the motor to the tank via first and second chamber passages to affect motion of the motor. The hydraulic control system may also have an accumulator, and an accumulator circuit configured to selectively direct fluid discharged from the motor to the accumulator for storage and to direct stored fluid from the accumulator to the motor to assist the motor. The hydraulic control system may further have a selector valve configured to selectively connect a higher pressure one of the first and second chamber passages with the accumulator, and a single pressure relief valve disposed within the accumulator circuit and configured to relief pressure from opposing sides of the motor.