A tape recorder has a base plate, at least one reel shaft, a head holder and a magnetic head. It further comprises a guide piece, a guide groove and at least one projection. The guide piece is made of a synthetic resin and mounted on the base plate (or the head holder). The guide groove is cut in the head holder (or the base plate) for receiving the guide piece to thereby restrict the direction in which the head holder moves. The projection is provided on the head holder (or the base plate). The end of the guide piece rides on the projection when the head holder is brought to a position where the magnetic head contacts a magnetic tape, whereby the projection urges the head holder against the base plate.