An input device is shared between a first remote desktop having a first display image, which is displayed on a first display of a first client device, and a second remote desktop having a second display image, which is displayed on a second display of a second client device. Upon detecting that an input pointer of the input device currently displayed within the first display image on the first display, is moved across an edge of the first remote desktop that corresponds to a virtual boundary between the first remote desktop and the second remote desktop, instructions are issued to a first remote computing device that is hosting the first remote desktop to discontinue receiving inputs made with the input device and a second remote computing device that is hosting the second remote desktop to begin receiving inputs made with the input device.