The described embodiments include a translation lookaside buffer (“TLB”) that is used for performing virtual address to physical address translations when making memory accesses in a memory in a computing device. In the described embodiments, the TLB includes a hierarchy of tables that are each used for performing virtual address to physical address translations based on the arrangement of pages of memory in corresponding regions of the memory. When performing a virtual address to physical address translation, the described embodiments perform a lookup in each of the tables in parallel for the virtual address to physical address translation and use a physical address that is returned from a lowest table in the hierarchy as the translation.