The present disclosure relates to systems, methods, and non-transitory computer-readable media that generate and implement behavioral policy graphs for digital image editing. For instance, in some embodiments, the disclosed systems generate, for a client device, a behavioral policy graph that assigns behaviors to object classes based on object relationships. The disclosed systems receive, from the client device, a digital image portraying a plurality of objects. Further, the disclosed systems determine behaviors of the plurality of objects utilizing the behavioral policy graph by determining, for each object of the plurality of objects, a behavior based on a relationship of the object with an additional object of the plurality of objects in accordance with the behavioral policy graph. The disclosed systems modify the digital image by modifying one or more objects based on the behaviors of the plurality of objects.