Techniques and systems are provided for dynamically adjusting virtual content provided by an extended reality system. In some examples, a system determines a level of distraction of a user of the extended reality system due to virtual content provided by the extended reality system. The system determines whether the level of distraction of the user due to the virtual content exceeds or is less than a threshold level of distraction, where the threshold level of distraction is determined based at least in part on one or more environmental factors associated with a real world environment in which the user is located. The system also adjusts one or more characteristics of the virtual content based on the determination of whether the level of distraction of the user due to the virtual content exceeds or is less than the threshold level of distraction.