An optical element with a phase structure configured to provide a phase difference between a first polarization component of light and a second polarization component of the light which is orthogonal to the first polarization component, is provided, in which the phase structure includes a first fine periodic structure which includes plural first dielectric plates with a first thickness and a first refractive index which are periodically arranged in a first fine period and a second fine periodic structure which includes plural second dielectric plates with a second thickness and a second refractive index which are periodically arranged in a second fine period, wherein a refractive index of a medium between the plural first dielectric plates and a refractive index of a medium between the plural second dielectric plates are different from the first refractive index and the second refractive index, and at least one of, the first fine period and the second fine period, a ratio of the first thickness to the first fine period and a ratio of the second thickness to the second fine period, and the first refractive index and the second refractive index, is different from each other.