The invention discloses an unmanned ship lateral control method based on a reinforcement learning algorithm, including the following steps: a dynamic model of an unmanned ship is built, and a simplified hull-path integrated model is used; a design scheme of an unmanned ship lateral controller is selected according to the requirement for unmanned ship lateral control performance; a reinforcement learning controller in the unmanned ship lateral controller is of an Actor-Critic structure, and is divided into an actor network and a critic network; a lateral deviation reference model of the unmanned ship is designed; and through performance index optimization, the system state or output follows the state of the reference model, and thus, the system performance is optimized. The advantage is as follows: the reinforcement learning is based on the 'trial and error' principle of the animal learning psychology, sequential decision making can be optimized according to evaluative feedback signals in the process of interaction between an unmanned ship and the environment, and thus, the method can be used to solve optimization control problems to which supervised learning can be hardly applied.