An organic electroluminescence display substrate and a fabrication method thereof, a display panel and a display device are provided. The organic electroluminescence display substrate includes a pixel array and a light extraction layer. The pixel array includes a plurality of pixels, each pixel includes a first sub-pixel emitting light of a first color, and each first sub-pixel includes an organic electroluminescence element. The light extraction layer covers on the pixel array, and at least includes a plurality of first light extraction units and a plurality of second light extraction units having different optical properties. A cathode light-emission side of each first sub-pixel is covered by at least one first light extraction unit and at least one second light extraction unit, or the cathode light-emission sides of two adjacent first sub-pixels are respectively covered by at least one first light extraction unit and at least one second light extraction unit.