A communication system includes: a decoding-probability module for calculating a decoding likelihood with a control unit for characterizing an alternative hypothesis regarding an arriving communication; a null-probability module, coupled to the decoding-probability module, for calculating a null likelihood for characterizing a null hypothesis regarding the arriving communication; a weight-calculation module, coupled to the decoding-probability module, for generating a decision weight corresponding to the decoding likelihood, the null likelihood, or a combination thereof; a reliability calculation module, coupled to the decoding-probability module, for calculating a decoding reliability with the decision weight, the decoding likelihood, and the null likelihood, the decoding reliability corresponding to a decoded-result; and a decoding module, coupled to the reliability calculation module, for decoding the arriving communication with a decoding parameter based on the decoding reliability for communicating with a device.