An MPI collective operation carried out in a fabric of network elements by transmitting MPI messages from all the initiator processes in an initiator node to designated ones of the responder processes in respective responder nodes. Respective payloads of the MPI messages are combined in a network interface device of the initiator node to form an aggregated MPI message. The aggregated MPI message is transmitted through the fabric to network interface devices of responder nodes, disaggregating the aggregated MPI message into individual messages, and distributing the individual messages to the designated responder node processes.