Messaging system with neural hair rendering

    公开(公告)号:US12272015B2

    公开(公告)日:2025-04-08

    申请号:US18653609

    申请日:2024-05-02

    Applicant: Snap Inc.

    Abstract: A messaging system performs neural network hair rendering for images provided by users of the messaging system. A method of neural network hair rendering includes processing a three-dimensional (3D) model of fake hair and a first real hair image depicting a first person to generate a fake hair structure, and encoding, using a fake hair encoder neural subnetwork, the fake hair structure to generate a coded fake hair structure. The method further includes processing, using a cross-domain structure embedding neural subnetwork, the coded fake hair structure to generate a fake and real hair structure, and encoding, using an appearance encoder neural subnetwork, a second real hair image depicting a second person having a second head to generate an appearance map. The method further includes processing, using a real appearance renderer neural subnetwork, the appearance map and the fake and real hair structure to generate a synthesized real image.

    Hair styles system for rendering hair strands based on hair spline data

    公开(公告)号:US12190422B2

    公开(公告)日:2025-01-07

    申请号:US18206985

    申请日:2023-06-07

    Applicant: Snap Inc.

    Abstract: Systems and methods are provided for rendering hair. The systems and methods include receiving hair spline data comprising coordinates of a plurality of hair strands; selecting a first hair strand of the plurality of hair strands; retrieving coordinates of the first hair strand; identifying based on the respective coordinates of the plurality of hair strands a second hair strand that is adjacent to the first hair strand; storing a reference to the second hair strand in association with the coordinates of the first hair strand; and generating one or more additional hair strands between the first hair strand and the second hair strand based on the coordinates of the first hair strand and the reference to the second hair strand.

    RAY TRACING BETWEEN AR AND REAL OBJECTS
    13.
    发明公开

    公开(公告)号:US20240303904A1

    公开(公告)日:2024-09-12

    申请号:US18179920

    申请日:2023-03-07

    Applicant: Snap Inc.

    CPC classification number: G06T15/06 G06T19/006

    Abstract: Aspects of the present disclosure involve a system for performing ray tracing between augmented reality (AR) and real-world objects. The system accesses, by the mobile device, a video depicting a first object. The system obtains, by the mobile device, a three-dimensional (3D) model of the first object. The system applies, by the mobile device, a ray tracing process to the 3D model of the first object to estimate an optical effect on a portion of the first object relative to a second object that is depicted in the video. The system modifies a visual property of the portion of the first object based on the optical effect relative to the second object.

    Augmented reality user interface control

    公开(公告)号:US10996811B2

    公开(公告)日:2021-05-04

    申请号:US16547045

    申请日:2019-08-21

    Applicant: Snap Inc.

    Abstract: Systems, devices, media, and methods are presented for controlling user interface with an object depicted within the user interface. The systems and methods initiates an augmented reality mode configured to present augmented reality elements within a graphical user interface. A face is detected within a field of view of an image capture component and presented within the graphical user interface. In response to detecting the face, the systems and methods sequentially present a set of augmented reality elements within the graphical user interface. A subset of the augmented reality elements and the face may be depicted contemporaneously. A movement is detected of at least a portion of the face relative to a first augmented reality element and presentation of the first augmented reality element is modified and at least one second augmented reality element is presented.

    MODIFYING MULTIPLE OBJECTS WITHIN A VIDEO STREAM

    公开(公告)号:US20210090347A1

    公开(公告)日:2021-03-25

    申请号:US17115096

    申请日:2020-12-08

    Applicant: Snap Inc.

    Abstract: Systems, devices, media, and methods are presented for presentation of modified objects within a video stream. The systems and methods receive a set of images within a video stream and identify at least a portion of a face in a first subset of images. The systems and methods determine face characteristics by analyzing the portion of the face in the first subset of images. The systems and methods apply a graphical representation of glasses to the face based on the face characteristics and cause presentation of a modified video stream including the portion of the face with the graphical representation of the glasses in a second subset of images of the set of images while receiving the video stream.

    Object modeling and replacement in a video stream

    公开(公告)号:US10810418B1

    公开(公告)日:2020-10-20

    申请号:US16122630

    申请日:2018-09-05

    Applicant: Snap Inc.

    Abstract: Systems, devices, and methods are presented for segmenting an image of a video stream with a client device by receiving one or more images depicting an object of interest and determining pixels within the one or more images corresponding to the object of interest. The systems, devices, and methods identify a position of a portion of the object of interest and determine a direction for the portion of the object of interest. Based on the direction of the portion of the object of interest, a histogram threshold is dynamically modified for identifying pixels as corresponding to the portion of the object of interest. The portion of the object of interest is replaced with a graphical interface element aligned with the direction of the portion of the object of interest.

    MESSAGING SYSTEM WITH NEURAL HAIR RENDERING

    公开(公告)号:US20230079136A1

    公开(公告)日:2023-03-16

    申请号:US17987285

    申请日:2022-11-15

    Applicant: Snap Inc.

    Abstract: A messaging system performs neural network hair rendering for images provided by users of the messaging system. A method of neural network hair rendering includes processing a three-dimensional (3D) model of fake hair and a first real hair image depicting a first person to generate a fake hair structure, and encoding, using a fake hair encoder neural subnetwork, the fake hair structure to generate a coded fake hair structure. The method further includes processing, using a cross-domain structure embedding neural subnetwork, the coded fake hair structure to generate a fake and real hair structure, and encoding, using an appearance encoder neural subnetwork, a second real hair image depicting a second person having a second head to generate an appearance map. The method further includes processing, using a real appearance renderer neural subnetwork, the appearance map and the fake and real hair structure to generate a synthesized real image.

Patent Agency Ranking