An all-to-all communication operation which is carried out in a fabric of networked entities by defining in each of the entities a plurality of memory regions of contiguous memory addresses holding messages therein, and exchanging the messages repeatedly with all the other entities. Relatively small messages are copied using a CPU and larger messages are transmitted using scatter/gather facilities.