This disclosure relates to a 5G or a pre-5G communication system to be provided to support a higher data rate following 4G communication systems such as LTE. A method according to one embodiment of the present invention is a method for attenuating interference of a signal received in a receiver of a filter bank multicarrier (FBMC) system, the method comprising the steps of: separately extracting data and a reference signal in a received FMBC symbol; obtaining a diagonal element channel of a desired symbol through a channel estimation from the extracted reference signal; generating an interference channel matrix of a non-diagonal component of the desired symbol, a diagonal component and a non-diagonal component of an interference symbol using a channel estimated diagonal component; reconfiguring to a banded channel matrix using an interference channel matrix; and attenuating the interference contained in the extracted data using the reconfigured banded channel matrix information and filter information of a transmitter of the filter bank multicarrier system.