A system and method of operating a vehicle using virtual vehicle controls, wherein the method includes: capturing image or video data from an area within an interior of the vehicle; sending a virtual vehicle control graphics request to a vehicle; receiving virtual vehicle control graphics response from the vehicle, wherein the virtual vehicle control graphics response includes virtual vehicle control graphics; and presenting the captured image or video data and the virtual vehicle control graphics on a display of the AR/VR device such that the virtual vehicle control graphics are presented over the captured image or video data, wherein the virtual vehicle control graphics includes one or more vehicle-user interface components.