公开(公告)号：US11783487B2

公开(公告)日：2023-10-10

申请号：US17567206

申请日：2022-01-03

Applicant: Snap Inc.

Inventor： Fedir Poliakov

IPC: G06T7/00 ,  G06T7/11 ,  G06T7/62 ,  G06T7/66 ,  G06T7/73 ,  G06T7/136 ,  G06V10/28 ,  G06V10/20 ,  G06V40/19 ,  G06V40/16 ,  G06V40/18 ,  G06T7/90

CPC classification number: G06T7/11 ,  G06T7/136 ,  G06T7/62 ,  G06T7/66 ,  G06T7/73 ,  G06T7/90 ,  G06V10/255 ,  G06V10/28 ,  G06V40/162 ,  G06V40/19 ,  G06V40/193 ,  G06V40/197 ,  G06T2207/10024 ,  G06T2207/30041

Abstract: Systems, devices, media, and methods are presented for gaze-based control of device operations. One method includes receiving a video stream from an imaging device, the video stream depicting one or more eyes, determining a gaze direction for the one or more eyes depicted in the video stream, detecting a change in the gaze direction of the one or more eyes, and triggering an operation in a client device based on the change in the gaze direction.

公开(公告)号：US11743426B2

公开(公告)日：2023-08-29

申请号：US16992968

申请日：2020-08-13

Applicant: Snap Inc.

Inventor： Lidiia Bogdanovych ,  William Brendel ,  Samuel Edward Hare ,  Fedir Poliakov ,  Guohui Wang ,  Xuehan Xiong ,  Jianchao Yang ,  Linjie Yang

IPC: G06T7/194 ,  G06V10/82 ,  H04N7/14 ,  G06T7/11 ,  G06N3/08 ,  G06N3/04 ,  G06V30/242 ,  G06F18/214 ,  G06F18/24 ,  G06V30/19 ,  H04N5/445 ,  H04N5/76

CPC classification number: H04N7/147 ,  G06F18/214 ,  G06F18/24765 ,  G06N3/04 ,  G06N3/08 ,  G06T7/11 ,  G06T7/194 ,  G06V10/82 ,  G06V30/19173 ,  G06V30/242 ,  G06T2207/10016 ,  G06T2207/10024 ,  G06T2207/20024 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/20221 ,  G06T2207/30201 ,  H04N5/44504 ,  H04N5/76 ,  H04N7/141

Abstract: A machine learning system can generate an image mask (e.g., a pixel mask) comprising pixel assignments for pixels. The pixels can he assigned to classes, including, for example, face, clothes, body skin, or hair. The machine learning system can be implemented. using a convolutional neural network that is configured to execute efficiently on computing devices having limited resources, such as mobile phones. The pixel mask can be used to more accurately display video effects interacting with a user or subject depicted in the image.

公开(公告)号：US20190354344A1

公开(公告)日：2019-11-21

申请号：US15981295

申请日：2018-05-16

Applicant: Snap Inc.

Inventor： Xin Chen ,  Yurii Monastyrshyn ,  Fedir Poliakov ,  Shubham Vij

IPC: G06F3/16 ,  G10L15/08 ,  G06F3/0482 ,  G06T11/00 ,  G06N3/08 ,  G06N3/04

Abstract: An audio control system can control interactions with an application or device using keywords spoken by a user of the device. The audio control system can use machine learning models (e.g., a neural network model) trained to recognize one or more keywords. Which machine learning model is activated can depend on the active location in the application or device. Responsive to detecting keywords, different actions are performed by the device, such as navigation to a pre-specified area of the application.

公开(公告)号：US10402689B1

公开(公告)日：2019-09-03

申请号：US15706057

申请日：2017-09-15

Applicant: Snap Inc.

Inventor： Lidiia Bogdanovych ,  William Brendel ,  Samuel Edward Hare ,  Fedir Poliakov ,  Guohui Wang ,  Xuehan Xiong ,  Jianchao Yang ,  Linjie Yang

IPC: G06T7/11 ,  G06K9/62 ,  G06K9/74 ,  G06T7/194 ,  G06N3/08 ,  G06N3/04 ,  G06K9/68 ,  H04N7/14 ,  H04N5/445 ,  H04N5/76

Abstract: A machine learning system can generate an image mask (e.g., a pixel mask) comprising pixel assignments for pixels. The pixels can be assigned to classes, including, for example, face, clothes, body skin, or hair. The machine learning system can be implemented using a convolutional neural network that is configured to execute efficiently on computing devices having limited resources, such as mobile phones. The pixel mask can be used to more accurately display video effects interacting with a user or subject depicted in the image.

公开(公告)号：US09830708B1

公开(公告)日：2017-11-28

申请号：US14884536

申请日：2015-10-15

Applicant: SNAP INC.

Inventor： Fedir Poliakov

IPC: G06K9/00 ,  G06T7/00 ,  G06T7/40

CPC classification number: G06T7/0081 ,  G06K9/00234 ,  G06K9/0061 ,  G06K9/3241 ,  G06K9/38 ,  G06T7/11 ,  G06T7/136 ,  G06T7/408 ,  G06T7/62 ,  G06T7/66 ,  G06T7/73 ,  G06T2207/10024 ,  G06T2207/30041

Abstract: Systems, devices, media, and methods are presented for segmenting an image of a video stream with a client device, binarizing an area of interest within one or more image, identifying an initial pupil location and an initial iris radius, and determining a final pupil location and a final iris radius. Some embodiments enable the client device to perform one or more operations within a user interface based on the image segmentation.

公开(公告)号：US12093607B2

公开(公告)日：2024-09-17

申请号：US17876842

申请日：2022-07-29

Applicant: Snap Inc.

Inventor： Xin Chen ,  Yurii Monastyrshyn ,  Fedir Poliakov ,  Shubham Vij

IPC: G06F3/16 ,  G06F3/0482 ,  G06N3/044 ,  G06N3/08 ,  G06T11/00 ,  G10L15/08 ,  G10L15/16

CPC classification number: G06F3/167 ,  G06F3/0482 ,  G06N3/044 ,  G06N3/08 ,  G06T11/001 ,  G10L15/08 ,  G10L2015/088 ,  G10L15/16

Abstract: An audio control system can control interactions with an application or device using keywords spoken by a user of the device. The audio control system can use machine learning models (e.g., a neural network model) trained to recognize one or more keywords. Which machine learning model is activated can depend on the active location in the application or device. Responsive to detecting keywords, different actions are performed by the device, such as navigation to a pre-specified area of the application.

公开(公告)号：US12075190B2

公开(公告)日：2024-08-27

申请号：US18221702

申请日：2023-07-13

Applicant: Snap Inc.

Inventor： Lidiia Bogdanovych ,  William Brendel ,  Samuel Edward Hare ,  Fedir Poliakov ,  Guohui Wang ,  Xuehan Xiong ,  Jianchao Yang ,  Linjie Yang

IPC: G06T7/11 ,  G06F18/214 ,  G06F18/24 ,  G06N3/04 ,  G06N3/08 ,  G06T7/194 ,  G06V10/82 ,  G06V30/19 ,  G06V30/242 ,  H04N7/14 ,  H04N5/445 ,  H04N5/76

CPC classification number: H04N7/147 ,  G06F18/214 ,  G06F18/24765 ,  G06N3/04 ,  G06N3/08 ,  G06T7/11 ,  G06T7/194 ,  G06V10/82 ,  G06V30/19173 ,  G06V30/242 ,  G06T2207/10016 ,  G06T2207/10024 ,  G06T2207/20024 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/20221 ,  G06T2207/30201 ,  H04N5/44504 ,  H04N5/76 ,  H04N7/141

Abstract: A machine learning system can generate an image mask (e.g., a pixel mask) comprising pixel assignments for pixels. The pixels can be assigned to classes, including, for example, face, clothes, body skin, or hair. The machine learning system can be implemented using a convolutional neural network that is configured to execute efficiently on computing devices having limited resources, such as mobile phones. The pixel mask can be used to more accurately display video effects interacting with a user or subject depicted in the image.

公开(公告)号：US20200098114A1

公开(公告)日：2020-03-26

申请号：US16698463

申请日：2019-11-27

Applicant: Snap Inc.

Inventor： Igor Kudriashov ,  Fedir Poliakov ,  Maksim Gusarov

IPC: G06T7/20 ,  G06K9/00 ,  G06K9/46 ,  G06T11/00 ,  G06T3/40 ,  G06T5/00 ,  G06T7/11 ,  G06T7/136 ,  G06T7/73 ,  G06T7/90

Abstract: Systems, devices, media, and methods are presented for segmenting an image of a video stream with a client device, identifying an area of interest, generating a modified area of interest within one or more image, identifying a first set of pixels and a second set of pixels, and modifying a color value for the first set of pixels.

公开(公告)号：US20200050866A1

公开(公告)日：2020-02-13

申请号：US16654898

申请日：2019-10-16

Applicant: Snap Inc.

Inventor： Samuel Edward Hare ,  Fedir Poliakov ,  Guohui Wang ,  Xuehan Xiong ,  Jianchao Yang ,  Linjie Yang ,  Shah Tanmay Anilkumar

IPC: G06K9/00 ,  G06T7/246 ,  G06T1/20

Abstract: A mobile device can generate real-time complex visual image effects using asynchronous processing pipeline. A first pipeline applies a complex image process, such as a neural network, to keyframes of a live image sequence. A second pipeline generates flow maps that describe feature transformations in the image sequence. The flow maps can be used to process non-keyframes on the fly. The processed keyframes and non-keyframes can be used to display a complex visual effect on the mobile device in real-time or near real-time.

公开(公告)号：US10535139B1

公开(公告)日：2020-01-14

申请号：US16418333

申请日：2019-05-21

Applicant: Snap Inc.

Inventor： Fedir Poliakov

IPC: G06K9/00 ,  G06T7/11 ,  G06T7/90

Abstract: Systems, devices, media, and methods are presented for gaze-based control of device operations. One method includes receiving a video stream from an imaging device, the video stream depicting one or more eyes, determining a gaze direction for the one or more eyes depicted in the video stream, detecting a change in the gaze direction of the one or more eyes, and triggering an operation in a client device based on the change in the gaze direction.