Systems and techniques are described for implementing autonomous control of earth-moving construction and/or mining vehicles, including to automatically determine and control autonomous movement (e.g., of a vehicle's hydraulic arm(s), tool attachment(s), tracks/wheels, rotatable chassis, etc.) to move materials or perform other actions based at least in part on data about an environment around the vehicle(s). A perception system on a vehicle that includes at least a LiDAR component may be used to repeatedly map a surrounding environment and determine a 3D point cloud with 3D data points reflecting the surrounding ground and nearby objects, with the LiDAR component mounted on a component part of the vehicle that is moved independently of the vehicle chassis to gather additional data about the environment. GPS data from receivers on the vehicle may further be used to calculate absolute locations of the 3D data points.