公开(公告)号：US10269177B2

公开(公告)日：2019-04-23

申请号：US15616634

申请日：2017-06-07

Applicant: Google Inc.

Inventor： Christian Frueh ,  Vivek Kwatra ,  Avneesh Sud

IPC: G06F3/01 ,  G06K9/00 ,  G06T7/73 ,  G02B27/01 ,  G06F17/30 ,  G06T15/04 ,  G06T15/40 ,  G06T17/00 ,  G06T17/20 ,  G06T19/00 ,  G06T19/20 ,  G06T7/246

Abstract: A camera captures an image of a user wearing a head mounted device (HMD) that occludes a portion of the user's face. A three-dimensional (3-D) pose that indicates an orientation and a location of the user's face in a camera coordinate system is determined. A representation of the occluded portion of the user's face is determined based on a 3-D model of the user's face. The representation replaces a portion of the HMD in the image based on the 3-D pose of the user's face in the camera coordinate system. In some cases, the 3-D model of the user's face is selected from 3-D models of the user's face stored in a database that is indexed by eye gaze direction. Mixed reality images can be generated by combining virtual reality images, unoccluded portions of the user's face, and representations of an occluded portion of the user's face.

公开(公告)号：US20180101984A1

公开(公告)日：2018-04-12

申请号：US15616604

申请日：2017-06-07

Applicant: Google Inc.

Inventor： Christian Frueh ,  Vivek Kwatra ,  Avneesh Sud

IPC: G06T17/20 ,  G06T19/20 ,  G06K9/00 ,  G06F17/30

CPC classification number: G06T17/205 ,  G02B27/0172 ,  G06F3/013 ,  G06F16/51 ,  G06F16/5838 ,  G06K9/00255 ,  G06K9/00288 ,  G06K9/00604 ,  G06T7/248 ,  G06T7/74 ,  G06T15/04 ,  G06T15/40 ,  G06T17/00 ,  G06T19/006 ,  G06T19/20 ,  G06T2200/04 ,  G06T2200/08 ,  G06T2207/10028 ,  G06T2207/30201 ,  G06T2207/30204 ,  G06T2219/2004 ,  G06T2219/2021

Abstract: A camera captures an image of a user wearing a head mounted device (HMD) that occludes a portion of the user's face. A three-dimensional (3-D) pose that indicates an orientation and a location of the user's face in a camera coordinate system is determined. A representation of the occluded portion of the user's face is determined based on a 3-D model of the user's face. The representation replaces a portion of the HMD in the image based on the 3-D pose of the user's face in the camera coordinate system. In some cases, the 3-D model of the user's face is selected from 3-D models of the user's face stored in a database that is indexed by eye gaze direction. Mixed reality images can be generated by combining virtual reality images, unoccluded portions of the user's face, and representations of an occluded portion of the user's face.

公开(公告)号：US20180101227A1

公开(公告)日：2018-04-12

申请号：US15616634

申请日：2017-06-07

Applicant: Google Inc.

Inventor： Christian Frueh ,  Vivek Kwatra ,  Avneesh Sud

IPC: G06F3/01 ,  G02B27/01 ,  G06T19/00

CPC classification number: G06T17/205 ,  G02B27/0172 ,  G06F3/013 ,  G06F17/3025 ,  G06F17/30256 ,  G06F17/3028 ,  G06K9/00255 ,  G06K9/00288 ,  G06K9/00604 ,  G06T7/248 ,  G06T7/74 ,  G06T15/04 ,  G06T15/40 ,  G06T17/00 ,  G06T19/006 ,  G06T19/20 ,  G06T2200/04 ,  G06T2200/08 ,  G06T2207/10028 ,  G06T2207/30201 ,  G06T2207/30204 ,  G06T2219/2004 ,  G06T2219/2021

Abstract: A camera captures an image of a user wearing a head mounted device (HMD) that occludes a portion of the user's face. A three-dimensional (3-D) pose that indicates an orientation and a location of the user's face in a camera coordinate system is determined. A representation of the occluded portion of the user's face is determined based on a 3-D model of the user's face. The representation replaces a portion of the HMD in the image based on the 3-D pose of the user's face in the camera coordinate system. In some cases, the 3-D model of the user's face is selected from 3-D models of the user's face stored in a database that is indexed by eye gaze direction. Mixed reality images can be generated by combining virtual reality images, unoccluded portions of the user's face, and representations of an occluded portion of the user's face.