Disclosed is a system and method for controlling a virtual engine sound of a hybrid vehicle or an electric vehicle. More specifically, the system includes a sensor for detecting when the vehicle is inside; a virtual engine sound generation portion for generating the virtual engine sound of the vehicle; and a control portion for receiving data detected by the sensor to control the virtual engine sound based upon whether or not the vehicle is inside, and transmitting a control signal corresponding to the data to the virtual engine generation portion.