A method of processing an audio signal is disclosed. The present invention includes receiving a downmix signal including at least one object signal and object information extracted when the downmix signal is generated, receiving mix information including mode selection information, the mix information for controlling the object signal, bypassing the downmix signal or extracting a background object and at least one independent object from the downmix signal, based on the mode selection information, and if the downmix signal is bypassed, generating multi-channel information using the object information and the mix information, wherein the downmix signal corresponds to a mono signal and wherein the mode selection information includes information indicating which one of modes including a normal mode, a mode for controlling the background object, and a mode for controlling the at least one independent object.