A messaging facility is described that enables the passing of packets of data from one processing element to another in a globally addressable, distributed memory multiprocessor without having an explicit destination address in the target processing elements memory. A message is a special cache-line-size write that has as its destination a pre-defined queue area in the memory of the receiving processing element. Arriving messages are placed in the queue in the order that they appear at the node by hardware queue management mechanisms. Flow control between processors is usually accomplished by the queue management hardware, with software intervention necessary to deal with the error cases caused by queue overflows, etc.