A method and apparatus of image data compression and decompression are disclosed. According to an embodiment of the present invention, the compression method partitions the image data into access units and encodes each access unit into a bitstream according to a target bit budget. Each access unit is encoded using first data compression to generate a first bitstream and the residual data is further encoded using second data compression to generate a second bitstream if the first bitstream is smaller than the target bit budget. In one example, the second data compression comprises bit plane coding applied to bit plane-ordered data, wherein the bit plane-ordered data is generated by scanning from a most significant bit to a least significant bit of the residual data in a bit plane-wise order. The decompression method comprises steps to recover reconstructed data from the first and second bitstreams.