A system and method is taught for data processing in an autonomous vehicle control system. Using information is acquired from the vehicle, network interface, and sensors mounted on the vehicle, the system can perceive situations around it with much less complexity in computation without losing crucial details, and then make navigation and control decisions. The system and method are operative to generate situation aware events, store them, and recall to predict situations for autonomous driving.