A zoom control device includes: a zoom magnification ratio change speed setting unit that sets a main image zoom magnification ratio change speed and a monitoring image zoom magnification ratio change speed according to a zoom operation by a user; and a zoom control unit that conducts a zoom control on a main image so that a zoom magnification ratio changes according to the main image zoom magnification ratio change speed, and conducts a zoom control on a monitoring image so that the zoom magnification ratio changes according to the monitoring image zoom magnification ratio change speed. The zoom magnification ratio change speed setting unit is configured to set the main image zoom magnification ratio change speed by smoothing the monitoring image zoom magnification ratio change speed.