A system is provided to manage on-chip memory access for multiple threads. The system comprises multiple parallel processing units to execute the threads, and an on-chip memory including multiple memory units and each memory unit includes a first region and a second region. The first region and the second region have different memory addressing schemes for parallel access by the threads. The system further comprises an address decoder coupled to the parallel processing units and the on-chip memory. The address decoder is operative to activate access by the threads to memory locations in the first region or the second region according to decoded address signals from the parallel processing units.