The disclosed systems and method correlates user behaviors with audio processing to achieve more accurate conclusions about sounds in a user's environment. These conclusions may, in turn, be used to adjust the way a device, such as AR glasses, operate or respond to the sounds. For example, audio events determined from processing speech can be correlated with behavior events determined by sensing a user to improve a speech-to-text transcript of the speech by separating, or otherwise altering, the text in the transcript by speaker.