An audio encoder, generating a stereo signal based on a multi-channel signal, includes a downmix unit for downmixing a multi-channel signal exceeding two channels to a two-channel stereo signal, a first coding unit for generating a first coded signal by coding the downmixed stereo signal, a second coding unit for generating a second coded signal by coding information to restore the downmixed stereo signal to a multi-channel signal, a code size calculating unit for calculating a code size of the second coded signal, and a first multiplexing unit for multiplexing the calculated code size in either the first coded signal or the second coded signal. Accordingly a decoder is able to easily extract a coded signal of the multi-channel signal based on the code size, and the decoder reproducing only the downmixed signal can be configured inexpensively.