Provided are a speech restoration system and method for concealing packet losses. The system includes a demultiplexer that demultiplexes an input bit stream and divides the input bit stream into several packets; a packet loss concealing unit that produces and outputs a linear spectrum pair (LSP) coefficient representing the vocal tract of voice and an excitation signal corresponding to a lost frame, when a packet loss occurs; and a speech restoring unit that synthesizes voice using the packets input from the demultiplexer, outputs the result as restored voice, and synthesizes voice corresponding to a lost packet using the LSP coefficient and the excitation signal input from the packet loss concealing unit and outputs the result as restored voice when the lost packet is detected, wherein the packet loss concealing unit repeats linear prediction coefficients (LPCs) of a last-received valid frame, produces a first excitation signal for the lost frame using a time scale modification (TSM) method, when the lost frame is voiceless, and produces a second excitation signal by re-estimating a gain parameter based on the first excitation signal, when the lost frame is voiced.