An iterative tree search-based preceding method for a multi-user Multi-Input Multi-Output (MIMO) communication system includes determining a reference value of a cumulative branch metric of a candidate symbol, eliminating candidates having values that exceed the determined reference value of the cumulative branch metric of the candidate symbol, and registering values, which do not exceed the determined reference value of the cumulative branch metric of the candidate symbol, as entries, and selecting the least value from the values registered as the entries. Thereby, the method has low complexity and similar performance compared to an existing Sphere Encoding (SE) technique.