A system and method for detecting construction site defects and hazards using artificial intelligence (AI) is provided. The system includes a movable base unit, a coordinate measurement scanner, a vision based sensor, and one or more processors. The one or more processors perform operations that include generating a two-dimensional (2D) map of the environment based at least in part on output from the coordinate measurement scanner, applying image recognition to the video stream data to identify and label a defect or hazard in the video data stream, correlating a location of the defect or hazard in the video stream data with the location in the 2D map, and recording the location of the defect or hazard in the 2D map.