公开(公告)号：US11363247B2

公开(公告)日：2022-06-14

申请号：US16792080

申请日：2020-02-14

Applicant: Valve Corporation

Inventor： Aaron Leiby

IPC: H04N13/139 ,  H04N13/161 ,  H04N19/182 ,  H04N19/517 ,  H04N13/194 ,  G06F3/01 ,  G06T17/20 ,  G06T19/00 ,  H04N13/00

Abstract: Described herein are motion smoothing techniques for a display, or display system, such as a head-mounted display (HMD), to account for motion of moving or animating objects in a way that mitigates judder. The display system may be separate from, yet communicatively coupled to, a host computer where a graphics-based application, such as a video game, is outputting frames for rendering on the display system. The host computer may generate motion vectors representing compressed pixel data for transmission to the display system. The motion vectors can be used by the display system to modify pixel data of a frame. The modified pixel data for the frame is “motion-smoothed” for rendering on the display system in a manner that mitigates judder of moving or animating objects.

公开(公告)号：US20190212812A1

公开(公告)日：2019-07-11

申请号：US16353422

申请日：2019-03-14

Applicant: Valve Corporation

Inventor： David Sawyer ,  Kenneth Birdwell ,  Kenneth Barnett ,  Tristan Reidford ,  Aaron Leiby ,  Emily Ridgway ,  Alexander C. Vlachos

IPC: G06F3/01 ,  G02B27/00 ,  G02B27/01

CPC classification number: G06F3/011 ,  G02B27/0093 ,  G02B27/017 ,  G02B2027/014 ,  G02B2027/0167

Abstract: Sensory feedback (“chaperoning”) systems and methods for guiding users in virtual/augmented reality environments such as walk-around virtual reality environments are described. Exemplary implementations assist with preventing collisions with objects in the physical operating space in which the user acts, among other potential functions and/or uses.

公开(公告)号：US11334145B2

公开(公告)日：2022-05-17

申请号：US16353422

申请日：2019-03-14

Applicant: Valve Corporation

Inventor： David Sawyer ,  Kenneth Birdwell ,  Kenneth Barnett ,  Tristan Reidford ,  Aaron Leiby ,  Emily Ridgway ,  Alexander C. Vlachos

IPC: G06F3/01 ,  G02B27/01 ,  G02B27/00

Abstract: Sensory feedback (“chaperoning”) systems and methods for guiding users in virtual/augmented reality environments such as walk-around virtual reality environments are described. Exemplary implementations assist with preventing collisions with objects in the physical operating space in which the user acts, among other potential functions and/or uses.

公开(公告)号：US20200160591A1

公开(公告)日：2020-05-21

申请号：US16748599

申请日：2020-01-21

Applicant: Valve Corporation

Inventor： Aaron Leiby ,  Alex Vlachos

IPC: G06T15/20 ,  G02B27/00 ,  G06T19/00 ,  G02B27/01

Abstract: Described herein are techniques for adjusting a prediction level and a throttle level, as frames are being rendered on a head-mounted display (HMD), based on an application's rendering performance. The prediction level is increased if a number of late frames, out of a past N rendered frames of (N being any suitable number), meets or exceeds a threshold number of late frames, which causes a compositor of the HMD to predict pose data of the HMD farther out into the future. The throttle level can be increased independently from, or in synchronization with, the increase in the prediction level to causes the compositor to throttle the frame rate of the application (e.g., to a fraction of the refresh rate of the HMD). The prediction level (and the throttle level, if at the same level) can be decreased if a particular number of consecutively-rendered frames finish rendering early.

公开(公告)号：US20210258555A1

公开(公告)日：2021-08-19

申请号：US16792080

申请日：2020-02-14

Applicant: Valve Corporation

Inventor： Aaron Leiby

IPC: H04N13/139 ,  H04N13/161 ,  H04N13/194 ,  H04N19/182 ,  H04N19/517

Abstract: Described herein are motion smoothing techniques for a display, or display system, such as a head-mounted display (HMD), to account for motion of moving or animating objects in a way that mitigates judder. The display system may be separate from, yet communicatively coupled to, a host computer where a graphics-based application, such as a video game, is outputting frames for rendering on the display system. The host computer may generate motion vectors representing compressed pixel data for transmission to the display system. The motion vectors can be used by the display system to modify pixel data of a frame. The modified pixel data for the frame is “motion-smoothed” for rendering on the display system in a manner that mitigates judder of moving or animating objects.

公开(公告)号：US10600236B2

公开(公告)日：2020-03-24

申请号：US16056277

申请日：2018-08-06

Applicant: Valve Corporation

Inventor： Aaron Leiby ,  Alex Vlachos

IPC: G06F1/16 ,  G06F3/14 ,  G06F3/0346 ,  G02B27/01 ,  G06T15/20 ,  G06T19/00 ,  G02B27/00

Abstract: Described herein are techniques for adjusting a prediction level and a throttle level, as frames are being rendered on a head-mounted display (HMD), based on an application's rendering performance. The prediction level is increased if a number of late frames, out of a past N rendered frames of (N being any suitable number), meets or exceeds a threshold number of late frames, which causes a compositor of the HMD to predict pose data of the HMD farther out into the future. The throttle level can be increased independently from, or in synchronization with, the increase in the prediction level to causes the compositor to throttle the frame rate of the application (e.g., to a fraction of the refresh rate of the HMD). The prediction level (and the throttle level, if at the same level) can be decreased if a particular number of consecutively-rendered frames finish rendering early.

公开(公告)号：US20200043223A1

公开(公告)日：2020-02-06

申请号：US16056277

申请日：2018-08-06

Applicant: Valve Corporation

Inventor： Aaron Leiby ,  Alex Vlachos

IPC: G06T15/20 ,  G02B27/01 ,  G02B27/00 ,  G06T19/00

Abstract: Described herein are techniques for adjusting a prediction level and a throttle level, as frames are being rendered on a head-mounted display (HMD), based on an application's rendering performance. The prediction level is increased if a number of late frames, out of a past N rendered frames of (N being any suitable number), meets or exceeds a threshold number of late frames, which causes a compositor of the HMD to predict pose data of the HMD farther out into the future. The throttle level can be increased independently from, or in synchronization with, the increase in the prediction level to causes the compositor to throttle the frame rate of the application (e.g., to a fraction of the refresh rate of the HMD). The prediction level (and the throttle level, if at the same level) can be decreased if a particular number of consecutively-rendered frames finish rendering early.

公开(公告)号：US11151776B2

公开(公告)日：2021-10-19

申请号：US16748599

申请日：2020-01-21

Applicant: Valve Corporation

Inventor： Aaron Leiby ,  Alex Vlachos

IPC: G06F1/16 ,  G06F3/14 ,  G06F3/0346 ,  G02B27/01 ,  G06T15/20 ,  G06T19/00 ,  G02B27/00

Abstract: Described herein are techniques for adjusting a prediction level and a throttle level, as frames are being rendered on a head-mounted display (HMD), based on an application's rendering performance. The prediction level is increased if a number of late frames, out of a past N rendered frames of (N being any suitable number), meets or exceeds a threshold number of late frames, which causes a compositor of the HMD to predict pose data of the HMD farther out into the future. The throttle level can be increased independently from, or in synchronization with, the increase in the prediction level to causes the compositor to throttle the frame rate of the application (e.g., to a fraction of the refresh rate of the HMD). The prediction level (and the throttle level, if at the same level) can be decreased if a particular number of consecutively-rendered frames finish rendering early.

公开(公告)号：US10733783B2

公开(公告)日：2020-08-04

申请号：US16155686

申请日：2018-10-09

Applicant: Valve Corporation

Inventor： Alex Vlachos ,  Aaron Leiby

IPC: G06T15/00 ,  G06T17/20 ,  G06T15/04 ,  G06T19/00 ,  H04N5/14

Abstract: Described herein are motion smoothing techniques for a display system to account for motion of moving or animating objects in a way that mitigates judder. For example, first pixel data and second pixel data associated with two previously-rendered frames may be provided to a graphics processing unit (GPU) as input. The video encoder of the GPU can process the input pixel data to generate an array of motion vectors which is used to modify third pixel data of a re-projected frame. The modified third pixel data for the re-projected frame is “motion-smoothed” for rendering on a display, such as a head-mounted display (HMD), in a manner that mitigates judder of moving or animating objects.

公开(公告)号：US20200111195A1

公开(公告)日：2020-04-09

申请号：US16155686

申请日：2018-10-09

Applicant: Valve Corporation

Inventor： Alex Vlachos ,  Aaron Leiby

IPC: G06T5/00 ,  G06T17/20 ,  H04N5/14 ,  G09G3/00 ,  G06T15/00 ,  G06T15/04 ,  G06T5/20

Abstract: Described herein are motion smoothing techniques for a display system to account for motion of moving or animating objects in a way that mitigates judder. For example, first pixel data and second pixel data associated with two previously-rendered frames may be provided to a graphics processing unit (GPU) as input. The video encoder of the GPU can process the input pixel data to generate an array of motion vectors which is used to modify third pixel data of a re-projected frame. The modified third pixel data for the re-projected frame is “motion-smoothed” for rendering on a display, such as a head-mounted display (HMD), in a manner that mitigates judder of moving or animating objects.