A system and method for highly automated driving of a vehicle to detect and classify pedestrians and traffic signs and other vehicles are provided. The system includes an on-vehicle camera for receiving image data. A location determining module is also on the vehicle to determine a location of the vehicle and a vehicle memory unit on the vehicle storing at least one particularized convolutional neural networks to process the image data. A vehicle processor is communicatively coupled to the vehicle memory unit and the camera and the location determining module and is configured to collect vehicle location data with the location determining module. The vehicle processor is also configured to process the image data using the at least one particularized convolutional neural network based the vehicle location data and environmental conditions around the vehicle to detect and classify pedestrians and traffic signs and other vehicles.