Disclosed approaches provide for irradiance caches which may be used to share irradiance between ray interactions spatially and/or temporally. An irradiance cache may store incoming irradiance or outgoing irradiance and may be updated by casting one or more rays from one or more locations to sample irradiance for the location(s). The number of rays that are cast may be reduced by ranking the locations, irradiance caches, and/or corresponding groups of geometry based on one or more characteristics thereof. For example, a ranking score may be computed based on camera distance, camera visibility, and/or a number of frames since a prior update. When sampling a location, outgoing irradiance from an outgoing irradiance cache may be used to determine shading when a hit distance of a ray used to generate the sample exceeds a threshold value.