A distance measurement device includes an emission unit, a detection unit, a first reduction unit that reduces, based on a detection result of the detection unit, influence of variation of an optical axis of the image formation optical system on a subject image received as light by a light receiving section, a second reduction unit that reduces variation of an optical axis of the directional light with respect to the subject based on the detection result of the detection unit, and a control unit that, in the case of operating the first reduction unit and the second reduction unit at the same time, controls the first reduction unit and the second reduction unit to reduce variation of an irradiation position of the directional light in the subject image received as light by the light receiving section.