A method of on-line diagnostic and prognostic assessment of an autonomous vehicle perception system includes detecting, via a sensor, a physical parameter of an object external to the vehicle. The method also includes communicating data representing the physical parameter via the sensor to an electronic controller. The method additionally includes comparing the data from the sensor to data representing the physical parameter generated by a geo-source model. The method also includes comparing results generated by a perception software during analysis of the data from the sensor to labels representing the physical parameter from the geo-source model. Furthermore, the method includes generating a prognostic assessment of a ground truth for the physical parameter of the object using the comparisons of the sensor data to the geo-source model data and of the software results to the geo-source model labels. A system for on-line assessment of the vehicle perception system is also disclosed.