A base member for swivel caster comprises a fixed base, a rotation base and a hollow staff. The fixed base includes a ring member, and a plurality of first raceways lathed in an inward surface of the ring member. A support cavity is defined through a center of the rotation base. Assembling holes are defined in a connecting portion and communicate with the support cavity and the second raceways. During assembly, a plurality of rolling balls are extended through the assembling holes and loaded into the first raceways and the second raceways. Then a plurality of biasing balls is loaded into the assembling holes. The staff is inserted into the support cavity and blocks the assembling holes. The rolling balls contact the biasing balls instead of fixed components whereby the rolling balls swivel and rotate smoothly.