3D object model reconstruction from 2D images

    公开(公告)号:US11688136B2

    公开(公告)日:2023-06-27

    申请号:US17249441

    申请日:2021-03-02

    Applicant: Snap Inc.

    Abstract: Systems and methods for reconstructing 3D models of human bodies from 2D images that counts for perspective and/or distortion effects are provided. The systems and methods include reconstructing a three-dimensional model of an object in a three-dimensional scene from a two-dimensional image comprising an image of the object. The systems and methods include determining an absolute depth of a key point of the object in the image; determining, using the absolute depth of the key point, a three-dimensional position of the key point in the three-dimensional scene; generating, using a neural network, a three-dimensional representation of the object, the three-dimensional representation comprising mesh nodes defined in a coordinate system relative to the key point; and positioning the three-dimensional representation of the object in the scene based on the position of the key point by applying a position dependent rotation to the three-dimensional object.

    GENERATIVE AI VIRTUAL CLOTHING TRY-ON

    公开(公告)号:US20250037333A1

    公开(公告)日:2025-01-30

    申请号:US18369935

    申请日:2023-09-19

    Applicant: Snap Inc.

    Abstract: An artificial intelligence (AI) network or neural network is trained, using a relatively small number of reference images of a target garment, to enable virtual clothing try-ons of the target garment. Example methods include determining a pose for a person depicted in an input image, determining an area of the input image to replace with a target garment, changing values of pixels within the area, and inputting the pose, the area, and a text prompt describing the target garment, into a neural network, to generate an output image, wherein the neural network is trained to generate the target garment. Example methods include training the neural network with images of clothing in a same class or category as the target garment to teach the neural network to shape the target garment in accordance with a pose of the person and to preserve other clothing and the background.

    AR body part tracking system
    13.
    发明授权

    公开(公告)号:US12198287B2

    公开(公告)日:2025-01-14

    申请号:US18376607

    申请日:2023-10-04

    Applicant: Snap Inc.

    Abstract: Aspects of the present disclosure involve a system for presenting AR items. The system performs operations including: receiving an image that includes a depiction of a first real-world body part in a real-world environment; applying a machine learning technique to the image to generate a plurality of dense outputs each associated with a respective pixel of a plurality of pixels in the image; applying a first task-specific decoder to the plurality of dense outputs to identify a pixel corresponding to a center of the first real-world body part; applying a second task-specific decoder using the identified pixel to retrieve a 3D rotation, translation and scale of first real-world body part from the plurality of dense outputs; modifying an AR object based on the 3D rotation, translation, and scale of first real-world body part; and modifying the image to include a depiction of the modified AR object.

    REAL-TIME TRY-ON USING BODY LANDMARKS
    14.
    发明公开

    公开(公告)号:US20240161242A1

    公开(公告)日:2024-05-16

    申请号:US18068383

    申请日:2022-12-19

    Applicant: Snap Inc.

    Abstract: Methods and systems are disclosed for transferring garments from one real-world object to another in real time using body landmarks. The system receives a first image that includes a depiction of a first person wearing a fashion item in a first pose. The system obtains a second image that includes a depiction of a second person in a second pose and generates a first set of body landmarks corresponding the first person in the first pose and a second set of body landmarks corresponding the second person wearing in the first pose. The system computes a deviation between the first set of body landmarks and the second set of body landmarks. The system generates a new image that depicts the second person wearing the fashion item worn by the first person based on the deviation between the first set of body landmarks and the second set of body landmarks.

Patent Agency Ranking