In an endoscope system according to an embodiment of the present invention, it is not necessary to touch an actual keyboard, mouse, or button to operate an endoscope apparatus, and thus it is not necessary to shift an instrument from one hand to the other hand or peel of/put on gloves in a medical setting. Accordingly, a necessary operation can be easily performed by using a line of sight and a gesture. In addition, it is not necessary to provide a keyboard, mouse, or button for operating the endoscope apparatus, and restrictions on the arrangement of the apparatus can be reduced. Furthermore, virtual objects corresponding to an apparatus to be operated and operation details are displayed in a pseudo manner, and thus the virtual object on which an operation is to be performed can be easily determined.