A simulator controlling device includes a swinging member for controlling a virtual moving object in the right and left direction. The swinging member is swung right and left within right and left limits. A first elastic portion forces and returns the swinging member to a neutral position within the right and left limits. A second elastic portion forces the swinging member toward the neutral position within right and left additional reaction areas. Each of the right and left additional reaction areas is formed from each of the right and left limits.