An apparatus for adjusting the height of a vehicle seat. The apparatus includes an input shaft for inputting a torque by using a lever installed at one side of the seat; an output member connected to a link means to output the torque for adjusting the height of the seat; a clutch drum interposed between the input shaft and the output member on a torque transfer path; a torque transfer unit comprising a plurality of neighboring engagers wherein the engagers disposed at two ends of the torque transfer unit are spaced apart from each other, and wedge-combined between the input shaft and the clutch drum to transfer the torque; and an elastic member for providing an elastic restoration force for moving the torque transfer unit to its initial position. According to the present invention, since a conventional retainer may be omitted, backlash caused by a retaining clearance and an operation clearance may be prevented and thus an operational efficiency may be increased. Also, an inversely input torque applied to an output member may be weakened and thus conventional problems of a reduction in operational efficiency and generation of noise may be solved.