The invention discloses an online following hardness measuring device for a steel wire knife production assembly line. The online following hardness measuring device comprises a support frame, whereintwo slide rails are arranged on the support frame; each slide rail is provided with a matching slide block; the slide block is connected with a first support base; a hardness measuring mechanism is arranged at the upper surface of the first support base; a screw nut is arranged at the lower surface of the first support base; a matching screw rod is arranged in the screw nut; both ends of the screw rod are respectively arranged on a front bearing support base and a back bearing support base, and are driven by a servo motor; the servo motor is arranged on the support frame; a photoelectric encoder is arranged in front of the first support base, and is arranged on the support frame; a synchronous guide wheel is arranged at one side of the photoelectric encoder. The online following hardnessmeasuring device has the advantages that by referring to the design concept of measurement along with carrier movement in the device type inertial navigation system, a following device system and a portable miniature hardness measuring instrument are skillfully combined, so that the online following hardness measuring device for a steel wire knife is creatively put forward.