-
公开(公告)号：US20210347053A1
公开(公告)日：2021-11-11
申请号：US17314866
申请日：2021-05-07
发明人： Ken Lee , Craig Cambias , Xin Hou
摘要： Described herein are methods and systems for providing virtual presence for telerobotics in a dynamic scene. A sensor captures frames of a scene comprising one or more objects. A computing device generates a set of feature points corresponding to objects in the scene and matches the set of feature points to 3D points in a map of the scene. The computing device generates a dense mesh of the scene and the objects using the matched feature points and transmits the dense mesh the frame to a remote viewing device. The remote viewing device generates a 3D representation of the scene and the objects for display to a user and receives commands from the user corresponding to interaction with the 3D representation of the scene. The remote viewing device transmits the commands to a robot device that executes the commands to perform operations on the objects in the scene.
-
公开(公告)号：US20190073825A1
公开(公告)日：2019-03-07
申请号：US16118894
申请日：2018-08-31
摘要： Described are methods and systems for enhancing depth sensor-based 3D geometry reconstruction with photogrammetry. A 3D sensor captures scans of a physical object, including related pose information and HD images corresponding to each scan. For each scan, a computing device generates an initial 3D model of the physical object, the initial model having missing sections. The computing device detects the missing sections in a 3D point cloud associated with the initial model and projects the missing sections in the point cloud to a corresponding HD image. The computing device generates image segments of the corresponding HD image that match the missing sections in the point cloud, generates a point cloud structure for each of the image segments of the corresponding HD image, and merges the initial model and the generated point cloud structures to generate a final 3D model with the missing sections filled in.
-
3.发明授权公开(公告)号：US09715761B2
公开(公告)日：2017-07-25
申请号：US14324891
申请日：2014-07-07
发明人： Ken Lee , Jun Yin , Xin Hou , Greg Werth , Yasmin Jahir
CPC分类号： G06T17/10 , G06K9/00201 , G06T7/55 , G06T7/75 , G06T17/20 , G06T2207/10028 , G06T2207/20028
摘要： Methods and systems are described for generating a three-dimensional (3D) model of a fully-formed object represented in a noisy or partial scene. An image processing module of a computing device receives images captured by a sensor. The module generates partial 3D mesh models of physical objects in the scene based upon analysis of the images, and determines a location of at least one target object in the scene by comparing the images to one or more 3D reference models and extracting a 3D point cloud of the target object. The module matches the 3D point cloud of the target object to a selected 3D reference model based upon a similarity parameter, and detects one or more features of the target object. The module generates a fully formed 3D model of the target object using partial or noisy 3D points from the scene, extracts the detected features of the target object and features of the 3D reference models that correspond to the detected features, and calculates measurements of the detected features.
-
公开(公告)号：US20190213773A1
公开(公告)日：2019-07-11
申请号：US16240404
申请日：2019-01-04
发明人： Ken Lee , Xiang Zhang , Xin Hou
CPC分类号： G06T13/40 , G06K9/00248 , G06T7/70 , G06T15/005 , G06T17/20 , H04L67/38
摘要： Described are methods and systems for real-time remote avatar creation and animation control. A sensor device captures images of non-rigid objects in a scene. A server coupled to the sensor device generates an initial 3D model for each of the non-rigid objects in the scene using the images. The server detects landmark points on the non-rigid objects using the initial 3D model. The server generates a control point animation map for the 3D model using the detected points. The server applies the animation map to the 3D model to generate a mapped 3D model. A viewer coupled to the server receives (i) the mapped 3D model and (ii) tracking information associated with the objects, including a model pose and deformation of the landmark points. The viewer modifies the mapped 3D model using the tracking information, and renders a video stream on the viewer using the modified 3D model.
-
公开(公告)号：US10192347B2
公开(公告)日：2019-01-29
申请号：US15596590
申请日：2017-05-16
摘要： Described herein are methods and systems for texturing a three-dimensional (3D) model using photogrammetry. A sensor captures scans of a physical object, including related pose information, and color images corresponding to each scan. A computing device generates a 3D mesh of the physical object. The computing device preprocesses the color images to remove blurry images and detect textured regions of the object in each non-blurry image. The computing device optimizes the pose information for each color image by generating associations between the color images and vertices in the 3D mesh and classifying the vertices as textured or non-textured. The computing device generates texture coordinates for the 3D mesh by segmenting the mesh, parameterizing the segments, and packing the parameterized segments into a texture atlas. The computing device paint the texture atlas using the color images that have optimized pose information to generate a model having texture coordinates for each vertex.
-
公开(公告)号：US20180101966A1
公开(公告)日：2018-04-12
申请号：US15726316
申请日：2017-10-05
发明人： Ken Lee , Yasmin Jahir , Xin Hou
CPC分类号： G06T7/75 , G06T3/0037 , G06T7/579 , G06T9/001 , G06T2207/10024 , G06T2207/10028 , G06T2207/30244 , G06T2219/024 , H04N21/23418 , H04N21/44012 , H04N21/6587 , H04N21/816
摘要： Described are methods and systems for generating a video stream of a scene including one or more objects. A sensor captures images of objects in a scene. A server coupled to the sensor, for each image, generates an initial 3D model for the objects and an initial 3D model of the scene. The server, for each image, captures pose information of the sensor as the sensor moves in relation to the scene or as the objects move in relation to the sensor. A viewing device receives the models and the pose information from the server. The viewing device captures pose information of the viewing device as the viewing device moves in relation to the scene. The viewing device renders a video stream on a display element using the received 3D models and at least one of the pose information of the sensor or the pose information of the viewing device.
-
公开(公告)号：US20160358382A1
公开(公告)日：2016-12-08
申请号：US15172723
申请日：2016-06-03
CPC分类号： H04N9/31 , G01B11/25 , G06F3/011 , G06F3/014 , G06F3/04815 , G06T17/00 , H04N9/3185 , H04N9/3194 , H04N13/117 , H04N13/246
摘要： Described herein are methods and systems for augmented reality in real-time using real-time 3D depth sensor and 3D projection techniques. A 3D sensor coupled to a computing device captures one or more scans of a physical object in a scene. The computing device generates a 3D model of the physical object based upon the one or more scans. The computing device determines a pose of the 3D model relative to a projector at the scene and predistorts image content based upon the pose of the 3D model to generate a rendered image map. A projector coupled to the computing device superimposes the rendered image map onto the physical object in the scene.
摘要翻译： 这里描述了使用实时3D深度传感器和3D投影技术实时增强现实的方法和系统。 耦合到计算设备的3D传感器捕获场景中的物理对象的一次或多次扫描。 计算设备基于一次或多次扫描生成物理对象的3D模型。 计算设备相对于场景处的投影仪确定3D模型的姿态，并且基于3D模型的姿态来预处理图像内容以生成渲染图像映射。 耦合到计算设备的投影仪将渲染的图像映射叠加到场景中的物理对象上。
-
8.发明授权公开(公告)号：US11620779B2
公开(公告)日：2023-04-04
申请号：US17139792
申请日：2020-12-31
发明人： Xiang Zhang , Xin Hou , Ken Lee , Yasmin Jahir
IPC分类号： G06T13/20 , G06T7/246 , G06T13/40 , G06T7/90 , G06T19/20 , G06T13/80 , G06V40/10 , G06V40/16
摘要： Described herein are methods and systems for remote visualization of real-time three-dimensional (3D) facial animation with synchronized voice. A sensor captures frames of a face of a person, each frame comprising color images of the face, depth maps of the face, voice data associated with the person, and a timestamp. The sensor generates a 3D face model of the person using the depth maps. A computing device receives the frames of the face and the 3D face model. The computing device preprocesses the 3D face model. For each frame, the computing device: detects facial landmarks using the color images; matches the 3D face model to the depth maps using non-rigid registration; updates a texture on a front part of the 3D face model using the color images; synchronizes the 3D face model with a segment of the voice data using the timestamp; and transmits the synchronized 3D face model and voice data to a remote device.
-
公开(公告)号：US20140086478A1
公开(公告)日：2014-03-27
申请号：US14039807
申请日：2013-09-27
IPC分类号： G06K9/36
CPC分类号： G06K9/36 , G06K9/00214 , G06K9/00973
摘要： Methods and apparatuses are described for processing 3D vision algorithms. A 3D vision processor device comprises one or more 3D vision processing cores. Each 3D vision processing core includes one or more memory blocks for storing location values associated with 3D point cloud images and an arithmetic logic unit coupled to the one or more memory modules. The arithmetic logic unit includes a plurality of memory registers for temporarily storing location values associated with a point in a 3D point cloud image and a processing unit coupled to the plurality of memory registers for performing arithmetic operations on the location values stored in the memory registers, the arithmetic operations used for 3D vision processing algorithms. The 3D vision processing core also includes a communication link for transferring data between the arithmetic logic unit and the memory modules.
摘要翻译： 描述了用于处理3D视觉算法的方法和装置。 3D视觉处理器设备包括一个或多个3D视觉处理核心。 每个3D视觉处理核心包括用于存储与3D点云图像相关联的位置值的一个或多个存储块，以及耦合到该一个或多个存储器模块的算术逻辑单元。 算术逻辑单元包括用于临时存储与3D点云图像中的点相关联的位置值的多个存储器寄存器和耦合到多个存储器寄存器的处理单元，用于对存储在存储器寄存器中的位置值进行算术运算， 用于3D视觉处理算法的算术运算。 3D视觉处理核心还包括用于在算术逻辑单元和存储器模块之间传送数据的通信链路。
-
10.发明申请公开(公告)号：US20210375020A1
公开(公告)日：2021-12-02
申请号：US17139792
申请日：2020-12-31
发明人： Xiang Zhang , Xin Hou , Ken Lee , Yasmin Jahir
摘要： Described herein are methods and systems for remote visualization of real-time three-dimensional (3D) facial animation with synchronized voice. A sensor captures frames of a face of a person, each frame comprising color images of the face, depth maps of the face, voice data associated with the person, and a timestamp. The sensor generates a 3D face model of the person using the depth maps. A computing device receives the frames of the face and the 3D face model. The computing device preprocesses the 3D face model. For each frame, the computing device: detects facial landmarks using the color images; matches the 3D face model to the depth maps using non-rigid registration; updates a texture on a front part of the 3D face model using the color images; synchronizes the 3D face model with a segment of the voice data using the timestamp; and transmits the synchronized 3D face model and voice data to a remote device.
-
-
-
-
-
-
-
-
-
搜索结果
23 条记录 (耗时 0.017 秒)
