A displacement is accurately measured at high speed to a measurement object having various surface states. In a displacement sensor including a confocal optical system in which an objective lens is moved along an optical axis, light emitted from a laser diode is formed into a slit beam by a cylindrical lens, a Y-axis side orthogonal to the optical axis is narrowed such that the light is collected on a surface of a measurement object, and an X-axis orthogonal to the optical axis is elongated in order to average a component of the light reflected from the surface. A photodiode receives the light reflected from the surface of the measurement object through an opening disposed in a position of conjugation with the laser diode. The opening is formed into a slit shape that is short in the Y-axis while being long in the X-axis. The displacement of the surface is measured from a position of the objective lens when a light receiving signal becomes the maximum.