An endoscope system includes: an endoscope; a moving device that holds and moves the endoscope; and at least one processor. The at least one processor is configured to: detect a movement direction of the endoscope moved by the moving device in a coordinate system of the moving device; estimate an area of a moving body in an image captured by the endoscope; detect a movement direction of an object in the image on the basis of another area in the image excluding the area of the moving body; calculate a deviation between a coordinate system of the endoscope and the coordinate system of the moving device on the basis of the movement direction of the endoscope and the movement direction of the object; and correct the coordinate system of the moving device on the basis of the deviation.