A method includes: sensing, by a mobile device when the mobile device is tethered to a head-mounted display (HMD) and when the mobile device is held by a first hand, first data; detecting, by the mobile device, information indicating which hand is used to perform a gesture from the first data; sending, by the mobile device, the information to the HMD; receiving, by the HMD, the information, and not detecting, by the HMD, the information by an image-based detection method; wherein the gesture is performed by a second hand and the information is an updated indication with respect to the gesture considering a hand switch state during a first elapsed duration from a sensing time of the first data to a sensing time of the first image data; and performing, by the HMD, side-adapted hand pose estimation on the first image data using the information.