An autofocus device includes: an imaging element; an imaging element driver; an imaging element driving controller; a detector; a focus determiner; and a speed parameter calculator. The focus determiner detects a focus position based on a contrast value. The speed parameter calculator calculates a speed parameter used to set a driving speed of the imaging element driver based on the contrast value. The imaging element driving controller performs first and second driving controls for detecting the focus position based on the speed parameter in an autofocus operation. Also, the imaging element driving controller sets, during the first driving control, a driving speed of the imaging element driver for the second driving control.