The locus of the displacement of the centers of the balls of a ball bearing is made elliptical and the radial bearing rigidity of the ball bearing in one radial direction is made greater than the radial bearing rigidity in another radial direction orthogonal to the one radial direction, or at least one of the axial length of the portion on the inner diametral surface of the inner race of the ball bearing which is fitted to a shaft and the axial length of the portion on the outer diametral surface of the outer race of the ball bearing which is fitted to a housing is made 1/2 or less of the bearing width of the ball bearing, whereby the radial rigidity of a ball bearing device is made very small as compared with the radial rigidity in the prior art. As a result, the stability of a swing arm system can be enhanced and higher speed and higher accuracy of control become possible.