A holographic display apparatus includes a spatial light modulator configured to generate hologram patterns to modulate light; an illuminator configured to emit the light to the spatial light modulator; and a controller configured to control operations of the spatial light modulator and the illuminator, the spatial light modulator being configured to generate, from among the hologram patterns, a first hologram pattern and a second hologram pattern according to the control operations of the controller, the first hologram pattern and the second hologram pattern being configured to form a first hologram image and a second hologram image having different viewpoints, and the controller being configured to set a first phase modulation value of the first hologram pattern and a second phase modulation value of the second hologram pattern to be different from each other such that hologram images having different viewpoints are formed.