In an apparatus for displaying a three-dimensional image, sub-pixels having color components are arranged in a longitudinal direction and in a lateral direction in a matrix form, in a display section for displaying a flat image, and a light ray control element is disposed so as to be opposed to the display section. In the light ray control element, linear optical openings extending in a vertical direction are arranged in the lateral direction. A sum of opening area lengths of opening areas of a plurality of sub-pixels adjacent to each other in a horizontal direction is varied in a single row, and the sum thereof in a plurality of rows becomes constant. Further, an arrangement of the sub-pixels is a color arrangement of a mosaic arrangement or a lateral stripe arrangement.