For a non-volatile memory that uses hard bit and a soft bit data in error correction operations, an on-the-fly compression scheme is used for the soft bit data. As soft bit data is transferred to a memory's input-output interface, the soft bit data is compressed prior to transmission to the an ECC engine memory controller, while hard bit data is transferred in un-compressed form.