A wearable audio output device in a physical environment includes an input device and one or more microphones. While ambient sound from the physical environment is being detected by the microphone(s), the wearable audio output device: while in a first mode, provides a first audio output including one or more pass-through audio components selected so as to increase audio pass-through of the ambient sound; detects an input via the input device; in response to detecting the input, and in accordance with a determination that the input is a first type of gesture, transitions from the first mode to a second mode; and, while in the second mode, provides a second audio output including one or more cancellation audio components selected so as to increase audio cancellation of the ambient sound.