Systems, apparatuses, and methods for scattering floating point values to heterogeneous memory devices are disclosed. An inference engine performs floating point calculations during pre-training and during post-training operations. A scatter unit stores the floating point number values in multiple memories with different error correction capabilities. A first portion of each floating point number value is stored in a first memory having a relatively high error correction capability, and a second portion of each floating point number value is stored in a second memory with a relatively low error correction capability. In one scenario, the first portion includes the sign and exponent fields, while the second portion includes the mantissa field. The resiliency of the inference engine to overcome small errors allows for convergence to the final result in spite of any errors in the retrieved second portion.