A system and method for providing multiple agents for decision making, trajectory planning, and control for autonomous vehicles are disclosed. A particular embodiment includes: partitioning a multiple agent autonomous vehicle control module for an autonomous vehicle into a plurality of subsystem agents, the plurality of subsystem agents including a deep computing vehicle control subsystem and a fast response vehicle control subsystem; receiving a task request from a vehicle subsystem; dispatching the task request to the deep computing vehicle control subsystem or the fast response vehicle control subsystem based on the content of the task request or a context of the autonomous vehicle; causing execution of the deep computing vehicle control subsystem or the fast response vehicle control subsystem by use of a data processor to produce a vehicle control output; and providing the vehicle control output to a vehicle control subsystem of the autonomous vehicle.