A Local illumination compensation system for video encoding and decoding uses memory for storing illumination compensation parameters and does not require access to reconstructed pixels of neighboring blocks. A set of illumination compensation parameters is stored in a dedicated buffer, which is of limited size, and which is decoupled from the coding unit level storage of information. The buffer contains a set of illumination compensation parameters, which may be, for example, computed (or determined in some other manner) on the fly or determined beforehand (for example for example obtained from the video signal or from a device).