Parallelism-aware scheduling of memory requests of threads in shared memory controllers. Parallel scheduling is achieved by prioritizing threads that already have requests being serviced in the memory banks. A first algorithm prioritizes requests of the last-scheduled thread that is currently being serviced. This is accomplished by tracking the thread that generated the last-scheduled request (if the request is still being serviced), and then scheduling another request from the same thread if there is an outstanding ready request from the same thread. A second algorithm prioritizes the requests of all threads that are currently being serviced. This is accomplished by tracking threads that have at least one request currently being serviced in the banks, and assigning the highest priority to these threads in the scheduling decisions. If there are no outstanding requests from any thread having requests that are being serviced, the algorithm defaults back to a baseline scheduling algorithm.