In one aspect, a system for determining subsurface soil layer characteristics as an agricultural implement is towed across a field by a work vehicle may include a RADAR sensor configured to capture data indicative of a subsurface soil layer characteristic of the field. Furthermore, the system may include a load sensor configured to capture data indicative of a load applied to the agricultural implement by soil within the field, with the load being indicative of the subsurface soil layer characteristic. Additionally, a controller of the system may be configured to determine a first value of the subsurface soil layer characteristic based on the data received from the RADAR and a second value of the subsurface soil layer characteristic based on the data received from the load sensor. Moreover, the controller may be configured to determine a differential between the first value and the second value.