Systems and methods are provided for controlling a vehicle. Control signals are generated at a high-level controller based on one or more sources of input data, comprising at least one of: sensors that provide sensor output information, map data and goals. The high-level controller comprises first controller modules comprising: an input processing module, a projection module, a memories module, a world model module, and a decision processing module that comprises a control model executor module. The control signals are processed at a low-level controller to generate commands that control a plurality of vehicle actuators of the vehicle in accordance with the control signals to execute one or more scheduled actions to be performed to automate driving tasks.