A method for determining residue coverage of a field includes receiving, with a computing system, a captured image depicting an imaged portion of a field from one or more imaging devices. Furthermore, the method includes determining, with the computing system, an image gradient orientation at each of a plurality of pixels within the captured image. Additionally, the method includes determining, with the computing system, a plurality of image gradient orientation densities associated with the captured image based on the determined image gradient orientations. Moreover, the method includes determining, with the computing system, a residue coverage parameter associated with the imaged portion of the field based on the determined plurality of image gradient orientation densities.