Data stored in non-volatile storage is decoded using iterative probabilistic decoding and multiple read operations to achieve greater reliability. An error correcting code such as a low density parity check code may be used. In one approach, initial reliability metrics, such as logarithmic likelihood ratios, are used in decoding read data of a set of non-volatile storage element. The decoding attempts to converge by adjusting the reliability metrics for bits in code words which represent the sensed state. If convergence does not occur, e.g., within a set time period, the state of the non-volatile storage element is sensed again, current values of the reliability metrics in the decoder are adjusted, and the decoding again attempts to converge. In another approach, the initial reliability metrics are based on multiple reads. Tables which store the reliability metrics and adjustments based on the sensed states can be prepared before decoding occurs.