-
公开(公告)号：US20190213773A1
公开(公告)日：2019-07-11
申请号：US16240404
申请日：2019-01-04
发明人： Ken Lee , Xiang Zhang , Xin Hou
CPC分类号： G06T13/40 , G06K9/00248 , G06T7/70 , G06T15/005 , G06T17/20 , H04L67/38
摘要： Described are methods and systems for real-time remote avatar creation and animation control. A sensor device captures images of non-rigid objects in a scene. A server coupled to the sensor device generates an initial 3D model for each of the non-rigid objects in the scene using the images. The server detects landmark points on the non-rigid objects using the initial 3D model. The server generates a control point animation map for the 3D model using the detected points. The server applies the animation map to the 3D model to generate a mapped 3D model. A viewer coupled to the server receives (i) the mapped 3D model and (ii) tracking information associated with the objects, including a model pose and deformation of the landmark points. The viewer modifies the mapped 3D model using the tracking information, and renders a video stream on the viewer using the modified 3D model.
-
2.发明申请公开(公告)号：US20210375020A1
公开(公告)日：2021-12-02
申请号：US17139792
申请日：2020-12-31
发明人： Xiang Zhang , Xin Hou , Ken Lee , Yasmin Jahir
摘要： Described herein are methods and systems for remote visualization of real-time three-dimensional (3D) facial animation with synchronized voice. A sensor captures frames of a face of a person, each frame comprising color images of the face, depth maps of the face, voice data associated with the person, and a timestamp. The sensor generates a 3D face model of the person using the depth maps. A computing device receives the frames of the face and the 3D face model. The computing device preprocesses the 3D face model. For each frame, the computing device: detects facial landmarks using the color images; matches the 3D face model to the depth maps using non-rigid registration; updates a texture on a front part of the 3D face model using the color images; synchronizes the 3D face model with a segment of the voice data using the timestamp; and transmits the synchronized 3D face model and voice data to a remote device.
-
3.发明授权公开(公告)号：US11620779B2
公开(公告)日：2023-04-04
申请号：US17139792
申请日：2020-12-31
发明人： Xiang Zhang , Xin Hou , Ken Lee , Yasmin Jahir
IPC分类号： G06T13/20 , G06T7/246 , G06T13/40 , G06T7/90 , G06T19/20 , G06T13/80 , G06V40/10 , G06V40/16
摘要： Described herein are methods and systems for remote visualization of real-time three-dimensional (3D) facial animation with synchronized voice. A sensor captures frames of a face of a person, each frame comprising color images of the face, depth maps of the face, voice data associated with the person, and a timestamp. The sensor generates a 3D face model of the person using the depth maps. A computing device receives the frames of the face and the 3D face model. The computing device preprocesses the 3D face model. For each frame, the computing device: detects facial landmarks using the color images; matches the 3D face model to the depth maps using non-rigid registration; updates a texture on a front part of the 3D face model using the color images; synchronizes the 3D face model with a segment of the voice data using the timestamp; and transmits the synchronized 3D face model and voice data to a remote device.
-
4.发明授权公开(公告)号：US11170552B2
公开(公告)日：2021-11-09
申请号：US16867196
申请日：2020-05-05
发明人： Xiang Zhang , Yasmin Jahir , Xin Hou , Ken Lee
摘要： Described herein are methods and systems for remote visualization of three-dimensional (3D) animation. A sensor of a mobile device captures scans of non-rigid objects in a scene, each scan comprising a depth map and a color image. A server receives a first set of scans from the mobile device and reconstructs an initial model of the non-rigid objects using the first set of scans. The server receives a second set of scans. For each scan in the second set of one or more scans, the server determines an initial alignment between the depth map and the initial model. The server converts the depth map into a coordinate system of the initial model, and determines a displacement between the depth map and the initial model. The server deforms the initial model to the depth map using the displacement, and applies a texture to at least a portion of the deformed model.
-
-
-
搜索结果
4 条记录 (耗时 0.013 秒)
