公开(公告)号：US11508167B2

公开(公告)日：2022-11-22

申请号：US16847009

申请日：2020-04-13

Applicant: Google LLC

Inventor： Boyang Deng ,  Kyle Genova ,  Soroosh Yazdani ,  Sofien Bouaziz ,  Geoffrey E. Hinton ,  Andrea Tagliasacchi

IPC: G06T17/00 ,  G06V20/64 ,  G06N3/08 ,  G06N3/04

Abstract: Methods, systems, and apparatus including computer programs encoded on a computer storage medium, for generating convex decomposition of objects using neural network models. One of the methods includes receiving an input that depicts an object. The input is processed using a neural network to generate an output that defines a convex representation of the object. The output includes, for each of a plurality of convex elements, respective parameters that define a position of the convex element in the convex representation of the object.

公开(公告)号：US20220326766A1

公开(公告)日：2022-10-13

申请号：US17301588

申请日：2021-04-08

Applicant: Google LLC

Inventor： Jason Todd Spencer ,  Seth Raphael ,  Sofien Bouaziz

IPC: G06F3/01 ,  G06K9/62

Abstract: A wearable computing device includes a frame, a camera mounted on the frame so as to capture images of an environment outside of the wearable computing device, a display device mounted on the frame so as to display the images captured by the camera, and at least one eye gaze tracking device mounted on the frame so as to track a gaze directed at the images displayed by the display device. In response to the detection of a fixation of the gaze on the display of images, the system may identify a pixel area corresponding to a fixation point of the fixation gaze on the display of images. The system may identify an object in the ambient environment corresponding to the identified pixel area, and set the identified object as a selected object for user interaction.

公开(公告)号：US11347320B1

公开(公告)日：2022-05-31

申请号：US17304986

申请日：2021-06-29

Applicant: Google LLC

Inventor： Dongeek Shin ,  David Kim ,  Sofien Bouaziz

IPC: G06F3/01 ,  G06F3/03

Abstract: A computing device, such as a wearable device, may include a gesture sensor that generates a gesture signal in response to a gesture of a user performed while the computing device is being worn or held by the user. A calibration sensor may generate a calibration signal characterizing a degree of tightness with which the computing device is being worn or held by the user. The gesture signal may be calibrated using the calibration signal, to obtain a calibrated gesture signal that is calibrated with respect to the degree of tightness. At least one function of the at least one computing device may be implemented, based on the calibrated gesture signal.

公开(公告)号：US11335023B2

公开(公告)日：2022-05-17

申请号：US15929811

申请日：2020-05-22

Applicant: Google LLC

Inventor： Sameh Khamis ,  Christian Haene ,  Hossam Isack ,  Cem Keskin ,  Sofien Bouaziz ,  Shahram Izadi

IPC: G06K9/00 ,  G06T7/73 ,  G06T3/40 ,  G06T11/00 ,  G06N3/08 ,  G06N3/04

Abstract: According to an aspect, a method for pose estimation using a convolutional neural network includes extracting features from an image, downsampling the features to a lower resolution, arranging the features into sets of features, where each set of features corresponds to a separate keypoint of a pose of a subject, updating, by at least one convolutional block, each set of features based on features of one or more neighboring keypoints using a kinematic structure, and predicting the pose of the subject using the updated sets of features.

公开(公告)号：US20220130111A1

公开(公告)日：2022-04-28

申请号：US17310678

申请日：2020-10-28

Applicant: Google LLC

Inventor： Ricardo Martin Brualla ,  Moustafa Meshry ,  Daniel Goldman ,  Rohit Kumar Pandey ,  Sofien Bouaziz ,  Ke Li

IPC: G06T17/20 ,  G06T15/04 ,  G06T7/40

Abstract: Systems and methods are described for utilizing an image processing system with at least one processing device to perform operations including receiving a plurality of input images of a user, generating a three-dimensional mesh proxy based on a first set of features extracted from the plurality of input images and a second set of features extracted from the plurality of input images. The method may further include generating a neural texture based on a three-dimensional mesh proxy and the plurality of input images, generating a representation of the user including at least a neural texture, and sampling at least one portion of the neural texture from the three-dimensional mesh proxy. In response to providing the at least one sampled portion to a neural renderer, the method may include receiving, from the neural renderer, a synthesized image of the user that is previously not captured by the image processing system.

公开(公告)号：US20210366146A1

公开(公告)日：2021-11-25

申请号：US15929811

申请日：2020-05-22

Applicant: Google LLC

Inventor： Sameh Khamis ,  Christian Haene ,  Hossam Isack ,  Cem Keskin ,  Sofien Bouaziz ,  Shahram Izadi

IPC: G06T7/73 ,  G06T3/40 ,  G06T11/00 ,  G06N3/04 ,  G06N3/08

Abstract: According to an aspect, a method for pose estimation using a convolutional neural network includes extracting features from an image, downsampling the features to a lower resolution, arranging the features into sets of features, where each set of features corresponds to a separate keypoint of a pose of a subject, updating, by at least one convolutional block, each set of features based on features of one or more neighboring keypoints using a kinematic structure, and predicting the pose of the subject using the updated sets of features.

公开(公告)号：US20210319209A1

公开(公告)日：2021-10-14

申请号：US16847009

申请日：2020-04-13

Applicant: Google LLC

Inventor： Boyang Deng ,  Kyle Genova ,  Soroosh Yazdani ,  Sofien Bouaziz ,  Geoffrey E. Hinton ,  Andrea Tagliasacchi

IPC: G06K9/00 ,  G06N3/04 ,  G06N3/08 ,  G06T17/00

Abstract: Methods, systems, and apparatus including computer programs encoded on a computer storage medium, for generating convex decomposition of objects using neural network models. One of the methods includes receiving an input that depicts an object. The input is processed using a neural network to generate an output that defines a convex representation of the object. The output includes, for each of a plurality of convex elements, respective parameters that define a position of the convex element in the convex representation of the object.

公开(公告)号：US12026833B2

公开(公告)日：2024-07-02

申请号：US17310678

申请日：2020-10-28

Applicant: Google LLC

Inventor： Ricardo Martin Brualla ,  Moustafa Meshry ,  Daniel Goldman ,  Rohit Kumar Pandey ,  Sofien Bouaziz ,  Ke Li

IPC: G06T17/20 ,  G06T7/40 ,  G06T15/04

CPC classification number: G06T17/20 ,  G06T7/40 ,  G06T15/04 ,  G06T2207/10028 ,  G06T2207/20081 ,  G06T2207/30201

Abstract: Systems and methods are described for utilizing an image processing system with at least one processing device to perform operations including receiving a plurality of input images of a user, generating a three-dimensional mesh proxy based on a first set of features extracted from the plurality of input images and a second set of features extracted from the plurality of input images. The method may further include generating a neural texture based on a three-dimensional mesh proxy and the plurality of input images, generating a representation of the user including at least a neural texture, and sampling at least one portion of the neural texture from the three-dimensional mesh proxy. In response to providing the at least one sampled portion to a neural renderer, the method may include receiving, from the neural renderer, a synthesized image of the user that is previously not captured by the image processing system.

公开(公告)号：US11995899B2

公开(公告)日：2024-05-28

申请号：US17302291

申请日：2021-04-29

Applicant: Google LLC

Inventor： Qinge Wu ,  Grant Yoshida ,  Catherine Boulanger ,  Erik Hubert Dolly Goossens ,  Cem Keskin ,  Sofien Bouaziz ,  Jonathan James Taylor ,  Nidhi Rathi ,  Seth Raphael

IPC: G06V20/62 ,  G02B27/00 ,  G02B27/01 ,  G06V10/20 ,  G06V30/10

CPC classification number: G06V20/63 ,  G02B27/0093 ,  G02B27/017 ,  G06V10/255 ,  G02B2027/0138 ,  G02B2027/014 ,  G06V30/10

Abstract: A head-mounted device (HMD) can be configured to determine a request for recognizing at least one content item included within content framed within a display of the HMD. The HMD can be configured to initiate a head-tracking process that maintains a coordinate system with respect to the content, and a pointer-tracking process that tracks a pointer that is visible together with the content within the display. The HMD can be configured to capture a first image of the content and a second image of the content, the second image including the pointer. The HMD can be configured to map a location of the pointer within the second image to a corresponding image location within the first image, using the coordinate system, and provide the at least one content item from the corresponding image location.

公开(公告)号：US11978268B2

公开(公告)日：2024-05-07

申请号：US17990532

申请日：2022-11-18

Applicant: Google LLC

Inventor： Boyang Deng ,  Kyle Genova ,  Soroosh Yazdani ,  Sofien Bouaziz ,  Geoffrey E. Hinton ,  Andrea Tagliasacchi

IPC: G06T17/00 ,  G06N3/045 ,  G06N3/08 ,  G06V20/64

CPC classification number: G06V20/64 ,  G06N3/045 ,  G06N3/08 ,  G06T17/00

Abstract: Methods, systems, and apparatus including computer programs encoded on a computer storage medium, for generating convex decomposition of objects using neural network models. One of the methods includes receiving an input that depicts an object. The input is processed using a neural network to generate an output that defines a convex representation of the object. The output includes, for each of a plurality of convex elements, respective parameters that define a position of the convex element in the convex representation of the object.