Neural network-based image stream modification

    公开(公告)号:US10679428B1

    公开(公告)日:2020-06-09

    申请号:US15990318

    申请日:2018-05-25

    Applicant: Snap Inc.

    Abstract: Systems, devices, media, and methods are presented for object detection and inserting graphical elements into an image stream in response to detecting the object. The systems and methods detect an object of interest in received frames of a video stream. The systems and methods identify a bounding box for the object of interest and estimate a three-dimensional position of the object of interest based on a scale of the object of interest. The systems and methods generate one or more graphical elements having a size based on the scale of the object of interest and a position based on the three-dimensional position estimated for the object of interest. The one or more graphical elements are generated within the video stream to form a modified video stream. The systems and methods cause presentation of the modified video stream including the object of interest and the one or more graphical elements.

    Encoding and decoding a stylized custom graphic

    公开(公告)号:US10657676B1

    公开(公告)日:2020-05-19

    申请号:US16022536

    申请日:2018-06-28

    Applicant: Snap Inc.

    Abstract: Disclosed are methods for encoding information in a graphic image. The information may be encoded so as to have a visual appearance that adopts a particular style, so that the encoded information is visually pleasing in the environment in which it is displayed. An encoder and decoder are trained during an integrated training process, where the encoder is tuned to minimize a loss when its encoded images are decoded. Similarly, the decoder is also trained to minimize loss when decoding the encoded images. Both the encoder and decoder may utilize a convolutional neural network in some aspects to analyze data and/or images. Once data is encoded, a style from a sample image is transferred to the encoded data. When decoding, the decoder may largely ignore the style aspects of the encoded data and decode based on a content portion of the data.

    Dense feature scale detection for image matching

    公开(公告)号:US10552968B1

    公开(公告)日:2020-02-04

    申请号:US15712990

    申请日:2017-09-22

    Applicant: Snap Inc.

    Abstract: Dense feature scale detection can be implemented using multiple convolutional neural networks trained on scale data to more accurately and efficiently match pixels between images. An input image can be used to generate multiple scaled images. The multiple scaled images are input into a feature net, which outputs feature data for the multiple scaled images. An attention net is used to generate an attention map from the input image. The attention map assigns emphasis as a soft distribution to different scales based on texture analysis. The feature data and the attention data can be combined through a multiplication process and then summed to generate dense features for comparison.

    NEURAL NETWORK FOR OBJECT DETECTION IN IMAGES

    公开(公告)号:US20190279046A1

    公开(公告)日:2019-09-12

    申请号:US16424404

    申请日:2019-05-28

    Applicant: Snap Inc.

    Abstract: Systems, devices, media, and methods are presented for identifying and categorically labeling objects within a set of images. The systems and methods receive an image depicting an object of interest, detect at least a portion of the object of interest within the image using a multilayer object model, determine context information, and identify the object of interest included in two or more bounding boxes.

Patent Agency Ranking