At an audio source, pause information is added to audio data, the combination of which is subsequently packetized. The resulting packets are transmitted to an audio destination via a network in which different packets may be subjected to varying levels of delay. At the audio destination, the pause information may be used to insert pauses at appropriate times to accommodate the occurrence of delays in packet delivery. In one embodiment, pauses are inserted based on a hierarchy of pause types. During pauses, audio filler information may be injected. In this manner, the effects of variable network delays upon reconstructed audio may be mitigated.