An example method for facilitating low latency remote direct memory access (RDMA) for microservers is provided and includes generating queue pair (QPs) in a memory of an input/output (I/O) adapter of a microserver chassis having a plurality of compute nodes executing thereon, the QPs being associated with a remote direct memory access (RDMA) connection between a first compute node and a second compute node in the microserver chassis, setting a flag in the QPs to indicate that the RDMA connection is local to the microserver chassis, and performing a loopback of RDMA packets within the I/O adapter from one memory region in the I/O adapter associated with the first compute node of the RDMA connection to another memory region in the I/O adapter associated with the second compute node of the RDMA connection.