Embodiments herein describe an integrated circuit that includes a NoC with at least two levels of switching: a sparse network and a non-blocking network. In one embodiment, the non-blocking network is a localized interconnect that provides an interface between the sparse network in the NoC and a memory system that requires additional bandwidth such as HBM2/3 or DDR5. Hardware elements connected to the NoC that do not need the additional benefits provided by the non-blocking network can connect solely to the sparse network. In this manner, the NoC provides a sparse network (which has a lower density of switching elements) for providing communication between lower bandwidth hardware elements and a localized non-blocking network for facilitating communication between the sparse network and higher bandwidth hardware elements.