A system and method for object detection and classification includes receiving by a controller a first data of a scene, wherein the data reflects the scene at a first time, and performing a first classification of at least one object within the scene based on the data. The method includes determining a projected location of the at least one object corresponding to an estimated location at a second time. The method includes receiving a second data of the scene reflecting the scene at the second time. The method includes determining whether the projected location of the object corresponds to the location of the object as reflected by the second data. The method includes determining whether performing a second classification of the object is necessary based on the determination of whether the projected location corresponds to the location of the object reflected by the second data.