A noise canceller removes a noise component from an input speech signal. The noise canceller includes a noise cancellation coefficient adjuster that adjusts a noise cancellation coefficient to determine an amount of noise cancellation. A noise spectrum storage device stores an estimated noise spectrum. A noise estimator estimates a noise spectrum by comparing an input spectrum with a noise spectrum stored in the noise spectrum storage device. A noise canceling/spectrum compensator subtracts the noise spectrum stored in the noise spectrum storage device from the input spectrum based on a coefficient acquired by the noise cancellation coefficient adjuster.