A three-dimensional holographic display device includes a light emitting diode (LED) array including a plurality of light sources controlled to sequentially output light according to a preset pattern, a lens configured to refract light incident from the LED array, a spatial light modulator (SLM) configured to modulate light incident from the lens, and a processor configured to generate a plurality of holographic signals each comprising depth information adjusted according to an arrangement location of each of the plurality of light sources, and for each of the plurality of light sources, control the SLM to modulate the light based on a holographic signal corresponding to the light source.