A roller holder includes a number of couplers disposed between two longitudinal rods for coupling the rods together and for forming a number of compartments to receive rolling members, and two retaining devices disposed in each compartment of the roller holder and attached to the rods for retaining the rolling members in the compartments and between the rods and the couplers of the roller holder, and for preventing the retaining devices from being impacted by the rolling members, the retaining devices each include a projected portion located above the rods and a protruded portion located below the rods and the couplers for engaging with the rolling members.