公开(公告)号：US20180253145A1

公开(公告)日：2018-09-06

申请号：US15969430

申请日：2018-05-02

Applicant: QUALCOMM Incorporated

Inventor： Ashwin SWAMINATHAN ,  Mahesh RAMACHANDRAN

IPC: G06F3/01 ,  G06K9/00 ,  G06F3/147 ,  G06K9/22 ,  G06F3/0481 ,  G06K9/32

Abstract: Methods and apparatus relating to enabling augmented reality applications using eye gaze tracking are disclosed. An exemplary method according to the disclosure includes displaying an image to a user of a scene viewable by the user, receiving information indicative of an eye gaze of the user, determining an area of interest within the image based on the eye gaze information, determining an image segment based on the area of interest, initiating an object recognition process on the image segment, and displaying results of the object recognition process.

公开(公告)号：US20150092985A1

公开(公告)日：2015-04-02

申请号：US14497135

申请日：2014-09-25

Applicant: QUALCOMM Incorporated

Inventor： Mahesh RAMACHANDRAN ,  Arvind RAMANANDAN ,  Christopher BRUNNER ,  Abhishek TYAGI ,  Murali Ramaswamy CHARI ,  Mingyang LI

IPC: G06T7/20

CPC classification number: G06T7/246 ,  G01C21/165 ,  G01S5/0263 ,  G01S5/0294 ,  G01S19/39 ,  G06T7/277 ,  G06T2207/10004 ,  G06T2207/10016

Abstract: Techniques are disclosed for estimating one or more parameters in a system. A device obtains measurements corresponding to a first set of features and a second set of features. The device estimates the parameters using an extended Kalman filter based on the measurements corresponding to the first set of features and the second set of features. The measurements corresponding to the first set of features are used to update the one or more parameters, and information corresponding to the first set of features. The measurements corresponding to the second set of features are used to update the parameters and uncertainty corresponding to the parameter. In on example, information corresponding to the second set of features is not updated during the estimating. Moreover, the parameters are estimated without projecting the information corresponding to the second set of features into a null-space.

Abstract translation: 公开了用于估计系统中的一个或多个参数的技术。 设备获得对应于第一组特征和第二组特征的测量。 该设备基于与第一组特征和第二组特征相对应的测量使用扩展卡尔曼滤波器来估计参数。 对应于第一组特征的测量用于更新一个或多个参数以及对应于第一组特征的信息。 对应于第二组特征的测量用于更新与参数对应的参数和不确定性。 在示例中，在估计期间不更新与第二组特征相对应的信息。 此外，估计参数而不将与第二组特征对应的信息投影到空空间中。

公开(公告)号：US20140267799A1

公开(公告)日：2014-09-18

申请号：US13843201

申请日：2013-03-15

Applicant: QUALCOMM INCORPORATED

Inventor： Shankar SADASIVAM ,  Ashwin SWAMINATHAN ,  Mahesh RAMACHANDRAN

IPC: H04N5/232

CPC classification number: H04N5/23245 ,  H04N5/23216

Abstract: Embodiments of the present invention are directed toward providing intelligent sampling strategies that make efficient use of an always-on camera. To do so, embodiments can utilize sensor information to determine contextual information regarding the mobile device and/or a user of the mobile device. A sampling rate of the always-on camera can then be modulated based on the contextual information.

Abstract translation: 本发明的实施例旨在提供能够有效利用始终在线的相机的智能采样策略。 为此，实施例可以利用传感器信息来确定关于移动设备和/或移动设备的用户的上下文信息。 然后可以基于上下文信息来调制始终在线摄像机的采样率。

公开(公告)号：US20200033943A1

公开(公告)日：2020-01-30

申请号：US16591382

申请日：2019-10-02

Applicant: QUALCOMM Incorporated

Inventor： Ashwin SWAMINATHAN ,  Mahesh RAMACHANDRAN

IPC: G06F3/01 ,  G06K9/00 ,  G06F3/147 ,  G06F3/0481 ,  G06K9/22

Abstract: Methods and apparatus relating to enabling augmented reality applications using eye gaze tracking are disclosed. An exemplary method according to the disclosure includes displaying an image to a user of a scene viewable by the user, receiving information indicative of an eye gaze of the user, determining an area of interest within the image based on the eye gaze information, determining an image segment based on the area of interest, initiating an object recognition process on the image segment, and displaying results of the object recognition process.

公开(公告)号：US20150094952A1

公开(公告)日：2015-04-02

申请号：US14497235

申请日：2014-09-25

Applicant: QUALCOMM Incorporated

Inventor： Mark Leo MOEGLEIN ,  Christopher BRUNNER ,  Arvind RAMANANDAN ,  Mahesh RAMACHANDRAN ,  Abhishek TYAGI ,  Murali Ramaswamy CHARI

IPC: G01C21/00 ,  G06T7/20 ,  G01S19/13

CPC classification number: G06T7/248 ,  G01C21/00 ,  G01C21/165 ,  G01C21/20 ,  G01C21/206 ,  G01C25/005 ,  G01S5/0263 ,  G01S5/0294 ,  G01S5/16 ,  G01S5/163 ,  G01S19/13 ,  G01S19/14 ,  G06K9/00671 ,  G06T7/579 ,  G06T7/60 ,  G06T7/80 ,  G06T2207/10004 ,  G06T2207/10016 ,  G06T2207/10032 ,  G06T2207/30212 ,  G06T2207/30241 ,  G06T2207/30244 ,  G06T2207/30252 ,  H04M2250/12 ,  H04M2250/52 ,  H04W4/029 ,  H04W4/043 ,  H04W4/33

Abstract: Embodiments disclosed pertain to the use of user equipment (UE) for the generation of a 3D exterior envelope of a structure based on captured images and a measurement set associated with each captured image. In some embodiments, a sequence of exterior images of a structure is captured and a corresponding measurement set comprising Inertial Measurement Unit (IMU) measurements, wireless measurements (including Global Navigation Satellite (GNSS) measurements) and/or other non-wireless sensor measurements may be obtained concurrently. A closed-loop trajectory of the UE in global coordinates may be determined and a 3D structural envelope of the structure may be obtained based on the closed loop trajectory and feature points in a subset of images selected from the sequence of exterior images of the structure.

Abstract translation: 所公开的实施例涉及使用用户设备（UE），用于基于所捕获的图像和与每个捕捉的图像相关联的测量集合来生成结构的3D外部包络。 在一些实施例中，捕获结构的外部图像的序列，并且包括惯性测量单元（IMU）测量，无线测量（包括全球导航卫星（GNSS）测量））和/或其他非无线传感器测量的对应测量集可以 同时获得。 可以确定UE在全局坐标中的闭环轨迹，并且可以基于从结构的外部图像的序列中选择的图像子集中的闭环轨迹和特征点来获得结构的3D结构包络。

公开(公告)号：US20140125700A1

公开(公告)日：2014-05-08

申请号：US13834207

申请日：2013-03-15

Applicant: QUALCOMM INCORPORATED

Inventor： Mahesh RAMACHANDRAN ,  Ashwin SWAMINATHAN

IPC: G09G5/377

CPC classification number: G09G5/377 ,  G06T7/55 ,  G06T2200/04 ,  G06T2200/08 ,  G06T2207/10016 ,  G06T2207/30244

Abstract: Systems and methods for performing localization and mapping with a mobile device are disclosed. In one embodiment, a method for performing localization and mapping with a mobile device includes identifying geometric constraints associated with a current area at which the mobile device is located, obtaining at least one image of the current area captured by at least a first camera of the mobile device, obtaining data associated with the current area via at least one of a second camera of the mobile device or a sensor of the mobile device, and performing localization and mapping for the current area by applying the geometric constraints and the data associated with the current area to the at least one image.

Abstract translation: 公开了用于利用移动设备执行定位和映射的系统和方法。 在一个实施例中，用于执行与移动设备的定位和映射的方法包括识别与移动设备所在的当前区域相关联的几何约束，获得由至少一个第一相机捕获的当前区域的至少一个图像 移动设备，通过移动设备的第二相机或移动设备的传感器中的至少一个获得与当前区域相关联的数据，以及通过应用几何约束和与该移动设备相关联的数据来执行当前区域的定位和映射 当前区域到至少一个图像。

公开(公告)号：US20150146926A1

公开(公告)日：2015-05-28

申请号：US14552183

申请日：2014-11-24

Applicant: QUALCOMM Incorporated

Inventor： Mahesh RAMACHANDRAN ,  Shankar SADASIVAM ,  Ashwin SWAMINATHAN

IPC: G06K9/00

CPC classification number: G06K9/00624 ,  G06F3/012 ,  G06T7/277 ,  G06T2200/04 ,  G06T2207/10016 ,  G06T2207/10028 ,  G06T2207/20076 ,  H04N5/23248 ,  H04N13/25 ,  H04N13/271

Abstract: Systems, apparatus and methods in a mobile device to enable and disable a depth sensor for tracking pose of the mobile device are presented. A mobile device relaying on a camera without a depth sensor may provide inadequate pose estimates, for example, in low light situations. A mobile device with a depth sensor uses substantial power when the depth sensor is enabled. Embodiments described herein enable a depth sensor only when images are expected to be inadequate, for example, accelerating or moving too fast, when inertial sensor measurements are too noisy, light levels are too low or high, an image is too blurry, or a rate of images is too slow. By only using a depth sensor when images are expected to be inadequate, battery power in the mobile device may be conserved and pose estimations may still be maintained.

Abstract translation: 提出了移动设备中的启用和禁用用于跟踪移动设备姿态的深度传感器的系统，设备和方法。 在没有深度传感器的相机上中继的移动设备可以提供不足的姿势估计，例如在低光情况下。 当深度传感器启用时，具有深度传感器的移动设备使用大量功率。 本文所描述的实施例仅在预期图像不足时才能使用深度传感器，例如，当惯性传感器测量太嘈杂时，加速或移动太快，光水平太低或过高，图像太模糊或速率 的图像太慢了。 通过在图像预期不足时仅使用深度传感器，移动设备中的电池电量可能被保存并且仍然可以维持姿态估计。

公开(公告)号：US20140327792A1

公开(公告)日：2014-11-06

申请号：US14268784

申请日：2014-05-02

Applicant: QUALCOMM Incorporated

Inventor： Alessandro MULLONI ,  Gerhard REITMAYR ,  Mahesh RAMACHANDRAN ,  Daniel WAGNER

IPC: H04N5/232

CPC classification number: H04N5/23296 ,  G01C25/005 ,  G06T7/73 ,  G06T7/80 ,  G06T2207/20092 ,  H04N5/23203

Abstract: Embodiments disclosed pertain to systems, method s and apparatus for the initialization of Computer Vision (CV) applications on user devices (UDs) comprising a camera and a display. In some embodiments, an optimal camera trajectory for initialization of a Computer Vision (CV) application may be determined based on an initial camera pose and an estimated pivot distance. For example, the initial camera pose may be estimated based on a first image captured by the camera. Further, the display may be updated in real-time with an indication of a desired movement direction for the camera. In some embodiments, the indication of desired movement direction may be based, in part, on a current camera pose and the optimal trajectory, where the current camera pose may be estimated based on a current image captured by the camera.

Abstract translation: 所公开的实施例涉及用于在包括相机和显示器的用户设备（UD）上初始化计算机视觉（CV）应用的系统，方法和装置。 在一些实施例中，用于初始化计算机视觉（CV）应用的最佳相机轨迹可以基于初始相机姿态和估计的枢转距离来确定。 例如，可以基于由相机拍摄的第一图像来估计初始相机姿态。 此外，可以用相机的期望的移动方向的指示来实时地更新显示。 在一些实施例中，期望移动方向的指示可以部分地基于当前摄像机姿态和最佳轨迹，其中当前摄像机姿态可以基于由照相机拍摄的当前图像来估计。

公开(公告)号：US20140104445A1

公开(公告)日：2014-04-17

申请号：US13651055

申请日：2012-10-12

Applicant: QUALCOMM INCORPORATED

Inventor： Mahesh RAMACHANDRAN ,  Christopher Brunner ,  Murali R. Chari ,  Serafin Diaz Spindola

IPC: H04N5/228

CPC classification number: G01C25/00 ,  G06T7/73 ,  G06T2207/10016 ,  G06T2207/30244

Abstract: An apparatus and method for generating parameters for an application, such as an augmented reality application (AR app), using camera pose and gyroscope rotation is disclosed. The parameters are estimated based on pose from images and rotation from a gyroscope (e.g., using least-squares estimation with QR factorization or a Kalman filter). The parameters indicate rotation, scale and/or non-orthogonality parameters and optionally gyroscope bias errors. In addition, the scale and non-orthogonality parameters may be used for conditioning raw gyroscope measurements to compensate for scale and non-orthogonality.

Abstract translation: 公开了一种使用相机姿态和陀螺仪旋转来生成诸如增强现实应用（AR app）的应用的参数的装置和方法。 基于来自图像的姿态和来自陀螺仪的旋转来估计参数（例如，使用具有QR因子分解的最小二乘估计或卡尔曼滤波器）。 参数表示旋转，刻度和/或非正交性参数以及可选的陀螺仪偏置误差。 此外，尺度和非正交性参数可用于调节原始陀螺仪测量以补偿比例和非正交性。