System and method for controlling the flow of messages in a computer system to minimize congestion and prevent deadlocks in communications. The computer system includes a main memory, a plurality of crossbar switches, a plurality of third level caches, and a plurality of input/output modules, which are interconnected via the communications network of the computer system. System and method prevents deadlocks between input/output modules and main memory, and between processors and main memory caused by data needed for making forward progress in processing being trapped behind messages. System and method utilize control signals and auxiliary buffers to hold and redirect messages out of the path of data so that data may flow to the input/output modules and processors when needed, and messages may flow when convenient.