An inertial drive actuator includes a shift unit that generates a shift in a first direction and in a second direction opposite to the first direction, a base plate that moves with the shift of the shift unit, and a mover disposed on a surface of the base plate and having a magnetic field generating unit. The mover has a first yoke that guides magnetic flux generated by the magnetic field generating unit such that the magnetic flux concentrates on a surface of the mover facing the base plate with respect to both S and N poles. Also included is a second yoke provided on a side of the base plate facing away from the mover. The frictional force acting between the mover and the base plate is controlled by controlling a magnetic field generated by the magnetic field generating unit to drive the mover.