A method of manufacturing a photo-alignment layer, includes: disposing a polymer material on a substrate; pre-baking the polymer material disposed on the substrate; irradiating a light to the pre-baked polymer material, to photo-align the pre-baked polymer material; and thermal-treating the irradiated pre-baked polymer material, to harden the irradiated pre-baked polymer material. The thermal-treating includes a first thermal-treatment, and a second thermal-treatment at a higher temperature than the first thermal-treatment.