An optical device comprises a shake detection unit adapted to detect shaking of the optical device, an amplifying unit adapted to amplify a difference between an output of the shake detection unit and a reference voltage, an offset calculation unit adapted to calculate an offset component based on an output of the amplifying unit, a changing unit adapted to change the reference voltage so as to remove the offset component, a correction unit adapted to correct the output of the amplifying unit so as to cancel out a change made by the changing unit, and a focal length detection unit adapted to detect a focal length of the imaging optical system, wherein the changing unit changes the reference voltage when the offset component exceeds a threshold, and the threshold is higher when the imaging optical system is on a telephoto side than on a wide-angle side.