A video driving game system includes a game console and a control device. The game console includes a processor unit for executing a driving game so as to manipulate a virtual driving vehicle and a display apparatus having a display screen for displaying the virtual driving vehicle. The control device includes a coupler member, a handheld handle projecting outward from the coupler member toward and be manipulated by an operator for turning the handle, thereby controlling the virtual driving vehicle, left and right distance detectors mounted respectively on left and right sides of the coupler member wherein, movement of body of the operator toward left and right sides of the coupler member causes the left and right detectors to generate first and second signals that command the virtual driving vehicle to swing leftwise and rightwise of the coupler member and movement of body of the operator toward the right side of the coupler member causes the right distance detector to generate a second signal that commands the virtual driving vehicle to swing rightwise of the coupler member.