A filter structure, a display substrate, a display panel and a display device are provided. The filter structure includes a first refractive index match layer, a waveguide layer, a second refractive index match layer and a grating layer, that are stacked, the waveguide layer is located between the first refractive index match layer and the second refractive index match layer, the second refractive index match layer is located between the waveguide layer (112) and the grating layer, and refractive index of the first refractive index match layer and refractive index of the second refractive index match layer both are smaller than refractive index of the waveguide layer.