SYSTEMS AND METHODS FOR HAND IMAGE SYNTHESIS

    公开(公告)号:WO2023069085A1

    公开(公告)日:2023-04-27

    申请号:PCT/US2021/055775

    申请日:2021-10-20

    IPC分类号: G06T15/00

    摘要: This application is directed to a method implemented at an electronic device for rendering realistic hand images. A computer system creates a three-dimensional (3D) rigged hand model having unique hand characteristics. The 3D rigged hand model is posed to a hand gesture based on one or more kinesiology parameters. The hand gesture of the 3D rigged hand model is adjusted with reference to one or more natural gestures, and rendered on a background image to obtain a hand image. In some embodiments, for each natural gesture, depth data are captured and measured from one or more marked key points of a respective subset of real hands that are posed with the respective natural gesture. A subset of the depth data of the real hands with the one or more natural gestures are transferred to the 3D rigged hand model.

    REALISTIC AUDIO DRIVEN 3D AVATAR GENERATION
    2.
    发明申请

    公开(公告)号:WO2022103877A1

    公开(公告)日:2022-05-19

    申请号:PCT/US2021/058838

    申请日:2021-11-10

    摘要: This application is directed to generation of a 3D avatar that is animated in synchronization with audio data. A computer system generates face parameters of a face associated with the person from an image. The face parameters include shape parameters describing a shape of the face and expression parameters describing an expression of the face. The computer system generates a color texture map and a displacement map of a 3D face model of the face associated with the person based on the face parameters. Additionally, audio-based face parameters are extracted from the audio data, independently of the image. In accordance with the shape parameters, expression parameters, color texture map, displacement map, and audio-based face parameters, the computer system renders the 3D avatar of the person in a video clip in which the 3D avatar is animated for an audio activity synchronous with the audio data.

    SYSTEM AND METHOD FOR DYNAMIC PORTRAIT RELIGHTING

    公开(公告)号:WO2023069086A1

    公开(公告)日:2023-04-27

    申请号:PCT/US2021/055776

    申请日:2021-10-20

    摘要: A method is implemented at an electronic device for relighting an input image. The electronic device reconstructs a 3D face model from a face image of the input image based on a plurality of facial landmarks, renders the 3D face model in a predefined lighting condition with a facial shadow and shading, blends the facial shadow and shading with the input image, and updates a non-facial portion of the input image with a style that matches the predefined lighting condition. In some embodiments, when the non-facial portion of the input image is updated, a color style of the non-facial portion of the input image is adjusted according to the predefined lighting condition and using a style-transfer neural network model. The predefined lighting condition corresponds to one of: leaf light, window light, contour light, morning light, dusk light, and arbitrary light.