According to embodiments of the invention, separate spatial indexes may be created which correspond to dynamic objects in a three dimensional scene and static objects in the three dimensional scene. By creating separate spatial indexes for static and dynamic objects, only the dynamic spatial index may need to be rebuilt in response to movement or changes in shape of objects in the three dimensional scene. Furthermore, the static and dynamic spatial indexes may be stored in separate portions of an image processing system's memory cache. By storing the static spatial index and the dynamic spatial index in separate portions of the memory cache, the dynamic portion of the memory cache may be updated without affecting the static portion of the spatial index in the memory cache.