A method and apparatus for determining the reliability of decoded data in a communication system. The method includes calculating a total sum of absolute values corresponding to Log Likelihood Ratio (LLR) values of received data, generating a first value obtained by multiplying the total sum of the absolute values by a predetermined threshold value, performing iterative decoding with respect to the LLR values of the received data, generating a survived path metric value having a maximum value among all path metric values as a decoded result and generating decoded data, comparing the first value with the survived path metric value, and determining whether the decoded data has suitable reliability according to the compared result.