A voice decoding apparatus capable of reducing degradation of quality of connection even silent compression is applied. The voice decoding apparatus comprises: a shaking absorption buffer temporarily accumulating received a packet and outputting the packet in a specified output; a background noise generation portion generating voice data of background noise according to background noise data included by the packet output by the shaking absorption buffer; a voice decoding portion decoding voice coding data included by the packet output by the shaking absorption buffer and generating voice data of voices; a speed conversion portion performing speed conversion for converting reproduction speed of the voice data after being decoded by the voice decoding portion; and a control portion controlling time duration of the background noise generated by the background noise generation portion and controlling the reproduction speed converted by the speed conversion portion on the basis of accumulating conditions of the packet in the shaking absorption buffer.