In one embodiment, a device in a network receives data indicative of a target state for one or more distributed learning agents in the network. The device determines a difference between the target state and state information maintained by the device regarding the one or more distributed learning agents. The device calculates a synchronization penalty score for each of the one or more distributed learning agents. The device selects a particular one of the one or more distributed learning agents with which to synchronize, based on the synchronization penalty score for the selected distributed learning agent and on the determined difference between the target state and the state information regarding the selected distributed learning agent. The device initiates synchronization of the state information maintained by the device regarding the selected distributed learning agent with state information from the selected distributed learning agent.