Detection of a signal in a receiver of a MIMO communication system includes a transmitter for signals transmission via K antennas and a receiver for receiving the signals via L reception antennas, such that L is greater than or equal to K and the system has a K×L-dimensional channel matrix, by converting the channel matrix into a plurality of matrixes having an upper triangle structure; dividing each of the matrixes into at least two sub-matrixes having a dimension lower than that of the channel matrix; detecting transmission symbols from corresponding antennas through decoding of a) lowest sub-matrix signal received which sub-matrix is constituted of components having only the channel characteristics of two antennas among the two sub-matrixes; and b) an upper sub-matrix signal using the transmission symbols; and outputting all of the detected transmission symbols, if transmission symbols by a highest sub-matrix among at least two sub-matrixes are detected.