Embodiments of the present invention relate to deterministic prediction in an image processing system. One aspect relates to an image processing system having a deterministic prediction decode unit for predicting individual pixels of an image based on a predetermined deterministic prediction algorithm. The deterministic prediction decode unit includes a look-up table, organized into four spatial phases, for storing values to be used by the predetermined deterministic prediction algorithm when converting a relatively low resolution image to a relatively higher resolution image. A prediction is made for a target pixel by accessing at least two of the four spatial phases of the look-up table to read at least two possible values of the target pixel. In one embodiment, the value of two target pixels can be provided within a same clock period, thus allowing for the decoding of two spatial phases with each access to the look-up table.