The present invention provides an OLED array substrate, a manufacturing method thereof, an OLED display panel and an OLED display device. The OLED array substrate comprises a substrate and a plurality of pixel units provided thereon, each pixel unit comprising a TFT, and a first electrode, an organic light-emitting layer, a second electrode and an optical coupling layer sequentially arranged on the TFT, wherein the optical coupling layer comprises a bottom contacting with the second electrode and an arched top protruding towards a light-exiting direction. By forming the optical coupling layer of each pixel unit on the OLED array substrate to be the arched top optical coupling layer, the OLED array substrate, the manufacturing method thereof, the OLED display panel and the OLED display device of the present invention attenuates total reflection of light inside the optical coupling layer, thereby improving light extraction efficiency.