公开(公告)号：US09724600B2

公开(公告)日：2017-08-08

申请号：US13281596

申请日：2011-10-26

申请人： Christopher Harley Willoughby ,  Jeffrey Jesus Evertt ,  Justin Avram Clark ,  Ben John Hindle ,  Peter Glenn Sarrett ,  Joel Deaguero

发明人： Christopher Harley Willoughby ,  Jeffrey Jesus Evertt ,  Justin Avram Clark ,  Ben John Hindle ,  Peter Glenn Sarrett ,  Joel Deaguero

IPC分类号： G06F1/10 ,  A63F13/40

CPC分类号： A63F13/40 ,  A63F13/10 ,  A63F2300/1087 ,  A63F2300/6045 ,  A63F2300/6607 ,  A63F2300/69

摘要： Methods, systems, and computer-storage media having computer-usable instructions embodied thereon, for controlling objects in a virtual environment are provided. Real-world objects may be received into a virtual environment. The real-world objects may be any non-human object. An object skeleton may be identified and mapped to the object. A user skeleton of the real-world user may also be identified and mapped to the object skeleton. By mapping the user skeleton to the object skeleton, movements of the user control the movements of the object in the virtual environment.

公开(公告)号：US09208571B2

公开(公告)日：2015-12-08

申请号：US13410907

申请日：2012-03-02

申请人： Jeffrey Jesus Evertt ,  Justin Avram Clark ,  Christopher Harley Willoughby ,  Mike Scavezze ,  Joel Deaguero ,  Relja Markovic ,  Joe Sola ,  David Haley

发明人： Jeffrey Jesus Evertt ,  Justin Avram Clark ,  Christopher Harley Willoughby ,  Mike Scavezze ,  Joel Deaguero ,  Relja Markovic ,  Joe Sola ,  David Haley

IPC分类号： H04N13/02 ,  G06T15/00 ,  G06T7/00 ,  H04N5/222 ,  G06T19/20

CPC分类号： G06T7/12 ,  A63F13/213 ,  A63F13/53 ,  A63F2300/1093 ,  A63F2300/6018 ,  G06T7/50 ,  G06T7/70 ,  G06T15/08 ,  G06T19/20 ,  G06T2200/08 ,  G06T2207/10004 ,  G06T2207/10028 ,  G06T2219/2012 ,  H04N5/2226 ,  H04N13/257 ,  H04N13/268 ,  H04N13/271

摘要： Digitizing objects in a picture is discussed herein. A user presents the object to a camera, which captures the image comprising color and depth data for the front and back of the object. For both front and back images, the closest point to the camera is determined by analyzing the depth data. From the closest points, edges of the object are found by noting large differences in depth data. The depth data is also used to construct point cloud constructions of the front and back of the object. Various techniques are applied to extrapolate edges, remove seams, extend color intelligently, filter noise, apply skeletal structure to the object, and optimize the digitization further. Eventually, a digital representation is presented to the user and potentially used in different applications (e.g., games, Web, etc.).

摘要翻译： 本文讨论了图像中的数字化对象。 用户将对象呈现给摄像机，该摄像机捕获包括对象前后的颜色和深度数据的图像。 对于前后图像，通过分析深度数据来确定最靠近相机的点。 从最接近的点，通过注意深度数据的大差异来发现对象的边缘。 深度数据也用于构建对象前后的点云结构。 应用各种技术来推断边缘，去除接缝，智能扩展颜色，过滤噪声，将骨架结构应用于对象，并进一步优化数字化。 最终，将数字表示呈现给用户并潜在地用于不同的应用（例如游戏，网络等）中。

公开(公告)号：US09336625B2

公开(公告)日：2016-05-10

申请号：US13280773

申请日：2011-10-25

申请人： Jeffrey Jesus Evertt ,  Justin Avram Clark ,  Christopher Harley Willoughby ,  Joel Deaguero ,  Relja Markovic

发明人： Jeffrey Jesus Evertt ,  Justin Avram Clark ,  Christopher Harley Willoughby ,  Joel Deaguero ,  Relja Markovic

IPC分类号： G06T17/00 ,  G06K9/00 ,  G06T7/00 ,  G06F17/30

CPC分类号： G06T17/00 ,  G06F17/30265 ,  G06K9/00201 ,  G06T7/521 ,  G06T2200/04 ,  G06T2200/08 ,  G06T2207/10024 ,  G06T2207/10028 ,  G06T2207/20212

摘要： Digitizing objects in a picture is discussed herein. A user presents the object to a camera, which captures the image comprising color and depth data for the front and back of the object. The object is recognized and digitized using color and depth data of the image. The user's client queries a server managing images uploaded by other users for virtual renditions of the object, as recognized in the other images. The virtual renditions from the other images are merged with the digitized version of the object in the image captured by the user to create a composite rendition of the object.

摘要翻译： 本文讨论了图像中的数字化对象。 用户将对象呈现给照相机，该照相机捕获包括对象前后的颜色和深度数据的图像。 使用图像的颜色和深度数据识别和数字化对象。 用户的客户端查询管理由其他用户上传的图像的服务器，以便在其他图像中识别出对象的虚拟渲染。 来自其他图像的虚拟渲染与由用户捕获的图像中的对象的数字化版本合并，以创建对象的复合再现。

公开(公告)号：US10796494B2

公开(公告)日：2020-10-06

申请号：US13186666

申请日：2011-07-20

申请人： Shawn C Wright ,  Jeffrey Jesus Evertt ,  Justin Avram Clark ,  Christopher Harley Willoughby ,  Mike Scavezze ,  Michael A Spalding ,  Kevin Geisner ,  Daniel L. Osborn

发明人： Shawn C Wright ,  Jeffrey Jesus Evertt ,  Justin Avram Clark ,  Christopher Harley Willoughby ,  Mike Scavezze ,  Michael A Spalding ,  Kevin Geisner ,  Daniel L. Osborn

IPC分类号： G06K1/00 ,  G06T1/00 ,  H04N1/00 ,  G06T19/20

摘要： A method, medium, and virtual object for providing a virtual representation with an attribute are described. The virtual representation is generated based on a digitization of a real-world object. Properties of the virtual representation, such as colors, shape similarities, volume, surface area, and the like are identified and an amount or degree of exhibition of those properties by the virtual representation is determined. The properties are employed to identify attributes associated with the virtual representation, such as temperature, weight, or sharpness of an edge, among other attributes of the virtual object. A degree of exhibition of the attributes is also determined based on the properties and their degrees of exhibition. Thereby, the virtual representation is provided with one or more attributes that instruct presentation and interactions of the virtual representation in a virtual world.

公开(公告)号：US09013489B2

公开(公告)日：2015-04-21

申请号：US13297890

申请日：2011-11-16

申请人： Jeffrey Jesus Evertt ,  Justin Avram Clark ,  Zachary Tyler Middleton ,  Matthew J Puls ,  Mark Thomas Mihelich ,  Dan Osborn ,  Andrew R Campbell ,  Charles Everett Martin ,  David M Hill

发明人： Jeffrey Jesus Evertt ,  Justin Avram Clark ,  Zachary Tyler Middleton ,  Matthew J Puls ,  Mark Thomas Mihelich ,  Dan Osborn ,  Andrew R Campbell ,  Charles Everett Martin ,  David M Hill

IPC分类号： G06T13/00 ,  A63F13/40 ,  G06T19/20 ,  G06T17/00 ,  G06K9/00

CPC分类号： A63F13/40 ,  A63F13/10 ,  A63F13/213 ,  A63F13/63 ,  A63F13/655 ,  A63F13/79 ,  A63F2300/1093 ,  A63F2300/5553 ,  G06K9/00281 ,  G06T19/20 ,  G06T2200/08

摘要： Systems, methods, and computer media for generating an avatar reflecting a player's current appearance. Data describing the player's current appearance is received. The data includes a visible spectrum image of the player, a depth image including both the player and a current background, and skeletal data for the player. The skeletal data indicates an outline of the player's skeleton. Based at least in part on the received data, one or more of the following are captured: a facial appearance of the player; a hair appearance of the player; a clothing appearance of the player; and a skin color of the player. A 3D avatar resembling the player is generated by combining the captured facial appearance, hair appearance, clothing appearance, and/or skin color with predetermined avatar features.

摘要翻译： 用于生成反映玩家当前外观的化身的系统，方法和计算机媒体。 收到描述玩家当前外观的数据。 数据包括播放器的可见光谱图像，包括播放器和当前背景的深度图像以及玩家的骨骼数据。 骨骼数据表示玩家骨架的轮廓。 至少部分地基于接收到的数据，捕获以下中的一个或多个：玩家的面部外观; 演员的头发外观; 玩家的服装外观; 和玩家的肤色。 类似于玩家的3D头像通过将捕获的面部外观，头发外观，服装外观和/或皮肤颜色与预定的化身特征组合而产生。

公开(公告)号：US20150145985A1

公开(公告)日：2015-05-28

申请号：US14091262

申请日：2013-11-26

申请人： Michael Jason Gourlay ,  Justin Avram Clark

发明人： Michael Jason Gourlay ,  Justin Avram Clark

IPC分类号： G06T7/00 ,  H04N7/18 ,  G06T11/20

CPC分类号： H04N7/183 ,  G06T7/246 ,  G06T7/33 ,  G06T17/00 ,  G06T2200/08 ,  G06T2207/10016 ,  G06T2207/10028 ,  G06T2207/20072 ,  G06T2207/30244

摘要： Depth maps of a physical space are obtained using a depth sensor carried by a rig such as a robot or a head mounted display device worn by a user. Visible light images are also obtained. The images and orientation readings are used to create a pose graph which includes nodes connected by links. The nodes are associated with different poses of the rig and the corresponding images. Links between the nodes represent correspondences between the images, and transforms between coordinate systems of the nodes. As new images are captured, the pose graph is updated to reduce an accumulation of errors. Furthermore, surfaces in the physical space can be reconstructed at any time according to the current state of the pose graph. Volumes used in a surface reconstruction process are anchored to the nodes such that the positions of the volumes are adjusted as the pose graph is updated.

摘要翻译： 物理空间的深度图是使用由诸如机器人的钻机所携带的深度传感器或由用户佩戴的头戴式显示装置获得的。 还可以获得可见光图像。 图像和方向读数用于创建包括通过链接连接的节点的姿态图。 节点与钻机的不同姿势和相应的图像相关联。 节点之间的链接表示图像之间的对应关系，并且在节点的坐标系之间进行变换。 当捕获新图像时，更新姿势图以减少错误的累积。 此外，物理空间中的表面可以根据姿势图的当前状态随时重建。 在表面重构过程中使用的体积被锚定到节点，使得当姿势图被更新时，体积的位置被调整。

公开(公告)号：US20150002507A1

公开(公告)日：2015-01-01

申请号：US13931507

申请日：2013-06-28

申请人： Anthony J. Ambrus ,  Jea Gon Park ,  Adam G. Poulos ,  Justin Avram Clark ,  Michael Jason Gourlay ,  Brian J. Mount ,  Daniel J. McCulloch ,  Arthur C. Tomlin

发明人： Anthony J. Ambrus ,  Jea Gon Park ,  Adam G. Poulos ,  Justin Avram Clark ,  Michael Jason Gourlay ,  Brian J. Mount ,  Daniel J. McCulloch ,  Arthur C. Tomlin

IPC分类号： G06T15/10 ,  G02B27/01

CPC分类号： G02B27/017 ,  G02B2027/0134 ,  G02B2027/014 ,  G02B2027/0178 ,  G06T19/00 ,  G06T19/003 ,  G06T2215/16

摘要： Technology is described for (3D) space carving of a user environment based on movement through the user environment of one or more users wearing a near-eye display (NED) system. One or more sensors on the near-eye display (NED) system provide sensor data from which a distance and direction of movement can be determined. Spatial dimensions for a navigable path can be represented based on user height data and user width data of the one or more users who have traversed the path. Space carving data identifying carved out space can be stored in a 3D space carving model of the user environment. The navigable paths can also be related to position data in another kind of 3D mapping like a 3D surface reconstruction mesh model of the user environment generated from depth images.

摘要翻译： 基于通过穿戴近眼显示（NED）系统的一个或多个用户的用户环境的移动来描述用于（3D）用户环境的空间雕刻技术。 靠近眼睛显示器（NED）系统上的一个或多个传感器提供传感器数据，从中可以确定运动的距离和方向。 可以基于用户高度数据和已经遍历路径的一个或多个用户的用户宽度数据来表示可导航路径的空间维度。 识别雕刻空间的空间雕刻数据可以存储在用户环境的3D空间雕刻模型中。 导航路径还可以与另一种3D映射中的位置数据相关联，如从深度图像生成的用户环境的3D表面重建网格模型。

公开(公告)号：US20160086385A1

公开(公告)日：2016-03-24

申请号：US14489802

申请日：2014-09-18

申请人： Michael Jason Gourlay ,  Justin Avram Clark

发明人： Michael Jason Gourlay ,  Justin Avram Clark

IPC分类号： G06T19/20 ,  G06T7/00

CPC分类号： G06T19/20 ,  G06T17/00 ,  G06T2207/10004 ,  G06T2207/20228 ,  G06T2219/2021

摘要： Volumes of a 3D physical space are used in a surface reconstruction process, where adjacent volumes share vertices so that no gaps or overlaps between the volumes exist. As a result, a continuous surface is obtained in the surface reconstruction process. The vertices are anchored to nodes in a pose graph, such that locations of the vertices are adjusted as the pose graph is updated. As a result, a deformation of the volumes is permitted. Based on the deformation of a volume, a region of a depth map of the physical space is deformed correspondingly. Each vertex can be anchored to a closest node of the pose graph, or to a point which is based on a combination of nodes. In one approach, the point is defined based on the closest node and other nodes within a defined radius of the closest node.

摘要翻译： 3D物理空间的体积用于表面重建过程，其中相邻体积共享顶点，使得存在体积之间不存在间隙或重叠。 结果，在表面重建处理中获得连续的表面。 顶点被锚定到姿势图中的节点，使得当姿势图被更新时，顶点的位置被调整。 结果，允许体积的变形。 基于体积的变形，物理空间的深度图的区域相应地变形。 每个顶点可以锚定到姿势图的最接近的节点，或者基于节点的组合的点。 在一种方法中，该点是基于最近节点和最近节点的定义半径内的其他节点来定义的。