Methods and apparatus to compile a software program to manage parallel μ caches are disclosed. In an example method, a compiler attempts to schedule a software program such that load instructions in a first set of load instructions has a first predetermine latency greater than the latency of the first cache. The compiler also marks a second set of load instructions with a latency less than the first predetermined latency to access the first cache. The compiler attempts to schedule the software program such that the load instruction in a third set have at least a second predetermined latency greater than the latency of the second cache. The compiler identifies a fourth set of load instructions in the scheduled software program having less than the second predetermined latency and marks the fourth set of load instructions to access the second cache.