HIDING LATENCY IN WIRELESS VIRTUAL AND AUGMENTED REALITY SYSTEMS

    公开(公告)号:US20210240257A1

    公开(公告)日:2021-08-05

    申请号:US16778767

    申请日:2020-01-31

    摘要: Systems, apparatuses, and methods for hiding latency for wireless virtual reality (VR) and augmented reality (AR) applications are disclosed. A wireless VR or AR system includes a transmitter rendering, encoding, and sending video frames to a receiver coupled to a head-mounted display (HMD). In one scenario, the receiver measures a total latency required for the system to render a frame and prepare the frame for display. The receiver predicts a future head pose of a user based on the total latency. Next, a rendering unit at the transmitter renders, based on the predicted future head pose, a new frame with a rendered field of view (FOV) larger than a FOV of the headset. The receiver rotates the new frame by an amount determined by the difference between the actual head pose and the predicted future head pose to generate a rotated version of the new frame for display.

    LATENCY MEASUREMENT SYSTEM AND METHOD
    2.
    发明公开

    公开(公告)号:US20240214283A1

    公开(公告)日:2024-06-27

    申请号:US18088935

    申请日:2022-12-27

    IPC分类号: H04N21/438 H04N21/442

    CPC分类号: H04N21/4384 H04N21/44209

    摘要: A client latency module generates a trigger event in response to an input event. The trigger event is inserted into an event queue to be sent to a content provider system. A stream including a plurality of images, audio data, or both, is received from the content provider system. A trigger event response generated in response to the trigger event is identified from the stream. A stream latency is determined by comparing a time corresponding to the trigger event with a time corresponding to the trigger event response. As a result, a single timer is used to measure latency of a streaming solution.

    REMOTE DISPLAY SYNCHRONIZATION TO PRESERVE LOCAL DISPLAY

    公开(公告)号:US20240108978A1

    公开(公告)日:2024-04-04

    申请号:US17955651

    申请日:2022-09-29

    摘要: A remote display synchronization technique preserves the presence of a local display device for a remotely-rendered video stream. A server and a client device cooperate to dynamically determine a target frame rate for a stream of rendered frames suitable for the current capacities of the server and the client device and networking conditions. The server generates from this target frame rate a synchronization signal that serves as timing control for the rendering process. The client device may provide feedback to instigate a change in the target frame rate, and thus a corresponding change in the synchronization signal. In this approach, the rendering frame rate and the encoding frequency may be “synchronized” in a manner consistent with the capacities of the server, the network, and the client device, resulting in generation, encoding, transmission, decoding, and presentation of a stream of frames that mitigates missed encoding of frames while providing acceptable latency.

    FRAME REPROJECTION FOR VIRTUAL REALITY AND AUGMENTED REALITY

    公开(公告)号:US20210192681A1

    公开(公告)日:2021-06-24

    申请号:US16719354

    申请日:2019-12-18

    IPC分类号: G06T3/00 G06T5/00 G06F3/01

    摘要: A client device for use in processing reality technology content is provided. The client device comprises memory configured to store data and a processor in communication with the memory. The processor is configured to receive, from a server device via network communication channel, a video stream comprising encoded video frames and motion vector information and extract, at the client device, the motion vector information. The processor is also configured to decode the video frames, determine whether to at least one of time warp the video frames and space warp the video frames using the extracted motion vector information. The processor is also configured to display the warped video frame data.

    Reducing latency in wireless virtual and augmented reality systems

    公开(公告)号:US11831888B2

    公开(公告)日:2023-11-28

    申请号:US17378323

    申请日:2021-07-16

    摘要: Systems, apparatuses, and methods for reducing latency for wireless virtual and augmented reality applications are disclosed. A virtual reality (VR) or augmented reality (AR) system includes a transmitter rendering, encoding, and sending video frames to a receiver coupled to a head-mounted display (HMD). In one scenario, rather than waiting until the entire frame is encoded before sending the frame to the receiver, the transmitter sends an encoded left-eye portion to the receiver while the right-eye portion is being encoded. In another scenario, the frame is partitioned into a plurality of slices, and each slice is encoded and then sent to the receiver while the next slice is being encoded. In a further scenario, each slice is being encoded while the next slice is being rendered. In a still further scenario, each slice is prepared for presentation by the receiver while the next slice is being decoded by the receiver.

    REDUCING LATENCY IN WIRELESS VIRTUAL AND AUGMENTED REALITY SYSTEMS

    公开(公告)号:US20210344939A1

    公开(公告)日:2021-11-04

    申请号:US17378323

    申请日:2021-07-16

    摘要: Systems, apparatuses, and methods for reducing latency for wireless virtual and augmented reality applications are disclosed. A virtual reality (VR) or augmented reality (AR) system includes a transmitter rendering, encoding, and sending video frames to a receiver coupled to a head-mounted display (HMD). In one scenario, rather than waiting until the entire frame is encoded before sending the frame to the receiver, the transmitter sends an encoded left-eye portion to the receiver while the right-eye portion is being encoded. In another scenario, the frame is partitioned into a plurality of slices, and each slice is encoded and then sent to the receiver while the next slice is being encoded. In a further scenario, each slice is being encoded while the next slice is being rendered. In a still further scenario, each slice is prepared for presentation by the receiver while the next slice is being decoded by the receiver.

    Reducing latency in wireless virtual and augmented reality systems

    公开(公告)号:US11070829B2

    公开(公告)日:2021-07-20

    申请号:US16716122

    申请日:2019-12-16

    摘要: Systems, apparatuses, and methods for reducing latency for wireless virtual and augmented reality applications are disclosed. A virtual reality (VR) or augmented reality (AR) system includes a transmitter rendering, encoding, and sending video frames to a receiver coupled to a head-mounted display (HMD). In one scenario, rather than waiting until the entire frame is encoded before sending the frame to the receiver, the transmitter sends an encoded left-eye portion to the receiver while the right-eye portion is being encoded. In another scenario, the frame is partitioned into a plurality of slices, and each slice is encoded and then sent to the receiver while the next slice is being encoded. In a further scenario, each slice is being encoded while the next slice is being rendered. In a still further scenario, each slice is prepared for presentation by the receiver while the next slice is being decoded by the receiver.