Apparatus and method for compressing an acceleration data structure such as a bounding volume hierarchy (BVH). For example, one embodiment of a graphics processing apparatus comprises: one or more cores to execute graphics instructions including instructions to perform ray tracing operations; and compression circuitry to compress lowest level nodes of a hierarchical acceleration data structure comprising a plurality of hierarchically arranged nodes, each of the lowest level nodes comprising pointers to leaf data; the compression circuitry to quantize the lowest level nodes to generate quantized lowest level nodes and to store each quantized lowest level node and associated leaf data without the pointers to the leaf data.