Degradation in image quality of color difference components is to be suppressed. Provided is an image compression device that performs fixed length compression of data to be compressed composed of a plurality of components including a luminance component. The image compression device includes a code amount allocation unit configured to determine, according to the luminance component, the code amount to be allocated to each of the plurality of components such that a total of the code amount allocated to each of the components remains constant, and a compression unit configured to compress each of the plurality of components in accordance with the code amount determined by the code amount allocation unit.