A retaining device includes a pair of annular retaining members and a pair of securing members. Each annular retaining member is adapted to be interposed between an axle and enlarged head portions of spoke tightening members in an annular space defined between the axle and an annular inner peripheral wall of a tubular mount member. The annular retaining member includes an annular inner ring portion sleeved on the axle, an annular outer ring portion surrounding and rotatable relative to the annular inner ring portion, and a rolling element disposed between the annular inner and outer ring portions. Each securing member is disposed to secure the annular inner ring portion on the axle such that the annular outer ring portion faces towards the enlarged head portions so as to guard shoulders of the spoke tightening members against being loosened from abutment with the inner peripheral wall.