The rotary cylinder block of a radial piston machine surrounds a control member whose outer surface has high pressure ports and low pressure ports communicating with the cylinders during rotation. The outer cylindrical surface of the control member also has a pair of circumferentially aligned first and second high pressure balancing grooves on either side of the high pressure ports and a pair of circumferentially aligned first and second low pressure balancing grooves on either side of the low pressure ports. The control member is formed with preferably straight diametrical ducts connecting the first and second high pressure balancing grooves with the first and second low pressure balancing grooves located on the same side of the high pressure and low pressure ports, respectively. The high pressure balancing grooves and the low pressure balancing grooves are filled, respectively, with high pressure fluid and low pressure fluid from the high pressure and low pressure ports, respectively, through the gap between the inner surface of the cylinder block and the outer surface of the control member so that the pressure fluid in the balancing grooves substantially centers the cylinder block on the control member.