In order to provide a surgical microscope system (100), a control apparatus (120), and a control method that enable position and orientation of a lens tube to be adjusted without requiring a large-scale system, the surgical microscope system includes an arm (112), a surgical microscope (113), a target value setting unit (122), an estimation unit (123), and a control unit (125), the arm includes a rotatable joint (118), the surgical microscope includes a microscope optical system (114) and a camera (115) that captures an operative field image that is a microscope magnification image of an operative field by the microscope optical system, the surgical microscope being supported by the arm, the target value setting unit sets target values of position and orientation of the surgical microscope, the estimation unit estimates the position and orientation of the surgical microscope on the basis of the operative field image and generates estimated values, and the control unit is configured to control a rotation of the joint in accordance with results of comparison of the target values with the estimated values.