Implementations are described herein for performing sparse depth estimation in the agricultural domain. In various implementations, one or more spatial aspects of a first visual annotation of a first candidate plant trait instance depicted in a first stereo image may be compared to one or more spatial aspects of a second visual annotation of a second candidate plant trait instance depicted in a second stereo image. Based on the comparison, it may be determined that the first and second candidate plant trait instances are a match for the same plant trait instance across the first and second stereo images. A disparity may be determined between the first and second candidate plant trait instances. Based on the disparity, a depth associated with the same plant trait instance may be estimated and used for various purposes.