公开(公告)号：US10474900B2

公开(公告)日：2019-11-12

申请号：US15706096

申请日：2017-09-15

Applicant: Snap Inc.

Inventor： Samuel Edward Hare ,  Fedir Poliakov ,  Guohui Wang ,  Xuehan Xiong ,  Jianchao Yang ,  Linjie Yang ,  Shah Tanmay Anilkumar

IPC: G06K9/00 ,  G06T7/246 ,  G06T1/20

Abstract: A mobile device can generate real-time complex visual image effects using asynchronous processing pipeline. A first pipeline applies a complex image process, such as a neural network, to keyframes of a live image sequence. A second pipeline generates flow maps that describe feature transformations in the image sequence. The flow maps can be used to process non-keyframes on the fly. The processed keyframes and non-keyframes can be used to display a complex visual effect on the mobile device in real-time or near real-time.

公开(公告)号：US10409569B1

公开(公告)日：2019-09-10

申请号：US15859004

申请日：2017-12-29

Applicant: Snap Inc.

Inventor： Guohui Wang ,  Fenglei Tian ,  Samuel Edward Hare ,  Sumant Hanumante ,  Tony Mathew

IPC: G06F9/44 ,  G06F8/41 ,  G06N3/04 ,  G06F9/50

Abstract: Among other things, embodiments of the present disclosure improve the functionality of computer software and systems by facilitating the automatic performance optimization of a software application based on the particular platform upon which the application runs. In some embodiments, the system can automatically choose a set of parameters or methods at run-time from a design space with pre-selected optimization methods and parameters (e.g., algorithms, software libraries, and/or hardware accelerators) for a specific task.

公开(公告)号：US12075190B2

公开(公告)日：2024-08-27

申请号：US18221702

申请日：2023-07-13

Applicant: Snap Inc.

Inventor： Lidiia Bogdanovych ,  William Brendel ,  Samuel Edward Hare ,  Fedir Poliakov ,  Guohui Wang ,  Xuehan Xiong ,  Jianchao Yang ,  Linjie Yang

IPC: G06T7/11 ,  G06F18/214 ,  G06F18/24 ,  G06N3/04 ,  G06N3/08 ,  G06T7/194 ,  G06V10/82 ,  G06V30/19 ,  G06V30/242 ,  H04N7/14 ,  H04N5/445 ,  H04N5/76

CPC classification number: H04N7/147 ,  G06F18/214 ,  G06F18/24765 ,  G06N3/04 ,  G06N3/08 ,  G06T7/11 ,  G06T7/194 ,  G06V10/82 ,  G06V30/19173 ,  G06V30/242 ,  G06T2207/10016 ,  G06T2207/10024 ,  G06T2207/20024 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/20221 ,  G06T2207/30201 ,  H04N5/44504 ,  H04N5/76 ,  H04N7/141

Abstract: A machine learning system can generate an image mask (e.g., a pixel mask) comprising pixel assignments for pixels. The pixels can be assigned to classes, including, for example, face, clothes, body skin, or hair. The machine learning system can be implemented using a convolutional neural network that is configured to execute efficiently on computing devices having limited resources, such as mobile phones. The pixel mask can be used to more accurately display video effects interacting with a user or subject depicted in the image.

公开(公告)号：US11847760B2

公开(公告)日：2023-12-19

申请号：US17714764

申请日：2022-04-06

Applicant: Snap Inc.

Inventor： Guohui Wang ,  Sumant Milind Hanumante ,  Ning Xu ,  Yuncheng Li

IPC: G06N3/08 ,  G06T3/40 ,  G06N3/04 ,  G06T1/20 ,  G06T11/60 ,  G06N3/063

CPC classification number: G06T3/4046 ,  G06N3/04 ,  G06N3/063 ,  G06N3/08 ,  G06T1/20 ,  G06T11/60 ,  G06T2207/20081

Abstract: Remote distribution of multiple neural network models to various client devices over a network can be implemented by identifying a native neural network and remotely converting the native neural network to a target neural network based on a given client device operating environment. The native neural network can be configured for execution using efficient parameters, and the target neural network can use less efficient but more precise parameters.

公开(公告)号：US11783448B2

公开(公告)日：2023-10-10

申请号：US17328689

申请日：2021-05-24

Applicant: Snap Inc.

Inventor： Shah Tanmay Anilkumar ,  Samuel Edward Hare ,  Guohui Wang

IPC: G06T3/40 ,  G06T5/50 ,  G06T7/11 ,  G06T7/174

CPC classification number: G06T3/4076 ,  G06T5/50 ,  G06T7/11 ,  G06T7/174 ,  G06T2207/10016 ,  G06T2207/20016 ,  G06T2207/20224

Abstract: Systems and methods are provided for initiating transfer of image data corresponding to at least one predetermined level of an image pyramid comprising higher resolution to a graphic processing unit (GPU) of the computing device, calculating, by the central processing unit (CPU) of the computing device, optical flow of at least one predetermined coarse level of the image pyramid, transferring, by the CPU of the computing device, the calculated optical flow of the at least one predetermined coarse level of the image pyramid to the GPU, calculating, by the GPU of the computing device, the optical flow of the at least one predetermined level of the image pyramid comprising higher resolution, and outputting, by the GPU of the computing device, the optical flow of the image data.

公开(公告)号：US20230024608A1

公开(公告)日：2023-01-26

申请号：US17816563

申请日：2022-08-01

Applicant: Snap Inc.

Inventor： Guohui Wang ,  Fenglei Tian ,  Samuel Edward Hare ,  Sumant Milind Hanumante ,  Tony Mathew

IPC: G06F8/41 ,  G06N3/04 ,  G06F9/50

Abstract: Among other things, embodiments of the present disclosure improve the functionality of computer software and systems by facilitating the automatic performance optimization of a software application based on the particular platform upon which the application runs. In some embodiments, the system can automatically choose a set of parameters or methods at run-time from a design space with pre-selected optimization methods and parameters (e.g., algorithms, software libraries, and/or hardware accelerators) for a specific task.

公开(公告)号：US10901714B1

公开(公告)日：2021-01-26

申请号：US16535935

申请日：2019-08-08

Applicant: Snap Inc.

Inventor： Guohui Wang ,  Fenglei Tian ,  Samuel Edward Hare ,  Sumant Hanumante ,  Tony Mathew

IPC: G06F9/44 ,  G06F8/41 ,  G06F9/50 ,  G06N3/04

Abstract: Among other things, embodiments of the present disclosure improve the functionality of computer software and systems by facilitating the automatic performance optimization of a software application based on the particular platform upon which the application runs. In some embodiments, the system can automatically choose a set of parameters or methods at run-time from a design space with pre-selected optimization methods and parameters (e.g., algorithms, software libraries, and/or hardware accelerators) for a specific task.

公开(公告)号：US20200050866A1

公开(公告)日：2020-02-13

申请号：US16654898

申请日：2019-10-16

Applicant: Snap Inc.

Inventor： Samuel Edward Hare ,  Fedir Poliakov ,  Guohui Wang ,  Xuehan Xiong ,  Jianchao Yang ,  Linjie Yang ,  Shah Tanmay Anilkumar

IPC: G06K9/00 ,  G06T7/246 ,  G06T1/20

Abstract: A mobile device can generate real-time complex visual image effects using asynchronous processing pipeline. A first pipeline applies a complex image process, such as a neural network, to keyframes of a live image sequence. A second pipeline generates flow maps that describe feature transformations in the image sequence. The flow maps can be used to process non-keyframes on the fly. The processed keyframes and non-keyframes can be used to display a complex visual effect on the mobile device in real-time or near real-time.

公开(公告)号：US20190087660A1

公开(公告)日：2019-03-21

申请号：US15706096

申请日：2017-09-15

Applicant: Snap Inc.

Inventor： Samuel Edward Hare ,  Fedir Poliakov ,  Guohui Wang ,  Xuehan Xiong ,  Jianchao Yang ,  Linjie Yang ,  Shah Tanmay Anilkumar

IPC: G06K9/00 ,  G06T7/246

Abstract: A mobile device can generate real-time complex visual image effects using asynchronous processing pipeline. A first pipeline applies a complex image process, such as a neural network, to keyframes of a live image sequence. A second pipeline generates flow maps that describe feature transformations in the image sequence. The flow maps can be used to process non-keyframes on the fly. The processed keyframes and non-keyframes can be used to display a complex visual effect on the mobile device in real-time or near real-time.

公开(公告)号：US20240249522A1

公开(公告)日：2024-07-25

申请号：US18624757

申请日：2024-04-02

Applicant: Snap Inc.

Inventor： Samuel Edward Hare ,  Fedir Poliakov ,  Guohui Wang ,  Xuehan Xiong ,  Jianchao Yang ,  Linjie Yang ,  Shah Tanmay Anilkumar

IPC: G06V20/40 ,  G06T1/20 ,  G06T7/246

CPC classification number: G06V20/40 ,  G06T1/20 ,  G06T7/248 ,  G06V20/46 ,  G06T2200/28 ,  G06T2207/10016 ,  G06T2207/20081

Abstract: A mobile device can generate real-time complex visual image effects using asynchronous processing pipeline. A first pipeline applies a complex image process, such as a neural network, to keyframes of a live image sequence. A second pipeline generates flow maps that describe feature transformations in the image sequence. The flow maps can be used to process non-keyframes on the fly. The processed keyframes and non-keyframes can be used to display a complex visual effect on the mobile device in real-time or near real-time.