公开(公告)号：US11810006B2

公开(公告)日：2023-11-07

申请号：US16951459

申请日：2020-11-18

Applicant: GM Global Technology Operations LLC

Inventor： Jaehoon Choe ,  Rajan Bhattacharyya ,  Kyungnam Kim ,  Kenji Yamada

IPC: G06N5/04 ,  B60W60/00 ,  G06V20/58 ,  B60W30/095 ,  G06N5/045

CPC classification number: G06N5/04 ,  B60W60/001 ,  B60W60/0011 ,  B60W60/00272 ,  B60W60/00276 ,  G06V20/58 ,  B60W30/0956 ,  B60W60/0027 ,  B60W2556/10 ,  G06N5/045

Abstract: A vehicle and a system and a method of operating the vehicle. The system includes a reasoning engine, an episodic memory, a resolver and a controller. The reasoning engine infers a plurality of possible scenarios based on a current state of an environment of the vehicle. The episodic memory determines a historical likelihood for each of the plurality of possible scenarios. The resolver selects a scenario from the plurality of possible scenarios using the historical likelihoods. The controller operates the vehicle based on the selected scenario.

公开(公告)号：US20220126861A1

公开(公告)日：2022-04-28

申请号：US17082193

申请日：2020-10-28

Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC

Inventor： Kenji Yamada ,  Kyungnam Kim ,  Rajan Bhattacharyya

IPC: B60W60/00 ,  G06N20/00

Abstract: A control system of the autonomous vehicle may generate multiple possible behavior control movements based on the driving goal and the assessment of the vehicle environment. In doing so, the method and system selects one of the best behavior control, among the multiple possible movements, and the selection is based on the quantitative grading of its driving behavior.

公开(公告)号：US20150294496A1

公开(公告)日：2015-10-15

申请号：US14251689

申请日：2014-04-14

Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC

Inventor： Swarup Medasani ,  Yuri Owechko ,  Kyungnam Kim

IPC: G06T15/08 ,  G06T7/00 ,  H04N7/18

CPC classification number: H04N7/181 ,  B25J9/1676 ,  G05B2219/40202 ,  G05B2219/40203 ,  G06K9/00771 ,  G06K2209/40 ,  G06T7/292 ,  G06T7/564 ,  G06T7/77 ,  G06T15/205 ,  G06T2207/30196 ,  G06T2207/30232

Abstract: A method of constructing a probabilistic representation of the location of an object within a workspace includes obtaining a plurality of 2D images of the workspace, with each respective 2D image being acquired from a camera disposed at a different location within the workspace. A foreground portion is identified within at least two of the plurality of 2D images, and each foreground portion is projected to each of a plurality of parallel spaced planes. An area is identified within each of the plurality of planes where a plurality of projected foreground portions overlap. These identified areas are combined to form a 3D bounding envelope of an object. This bounding envelope is a probabilistic representation of the location of the object within the workspace.

Abstract translation: 构建工作空间内对象位置的概率表示的方法包括获得工作空间的多个2D图像，其中每个相应的2D图像是从设置在工作空间内的不同位置处的相机获取的。 在多个2D图像的至少两个中识别前景部分，并且将每个前景部分投影到多个平行的间隔平面中的每一个。 在多个投影的前景部分重叠的多个平面的每一个中识别出一个区域。 这些识别的区域被组合以形成对象的3D边界包络。 该边界包络是工作空间内对象位置的概率表示。

公开(公告)号：US11873006B2

公开(公告)日：2024-01-16

申请号：US17026522

申请日：2020-09-21

Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC

Inventor： Tiffany J. Hwu ,  Rajan Bhattacharyya ,  Michael J. Daily ,  Kyungnam Kim

IPC: B60W60/00 ,  G01C21/00 ,  G06N3/04 ,  B60W10/04 ,  G01C21/34 ,  G06N3/08 ,  B60W30/12

CPC classification number: B60W60/0011 ,  B60W10/04 ,  B60W30/12 ,  B60W60/0027 ,  G01C21/3407 ,  G01C21/3815 ,  G06N3/04 ,  G06N3/08

Abstract: A virtual lane estimation system includes a memory device, a sensor and a computer. The memory device is configured to store a road map that corresponds to a portion of a road ahead of a vehicle. The sensor is configured to observe a plurality of trajectories of a plurality of neighboring vehicles that traverse the portion of the road. The computer is configured to initialize a recursive self-organizing map as a plurality of points arranged as a two-dimensional grid aligned with the road map, train the points in the recursive self-organizing map in response to the trajectories, generate a directed graph that contains one or more virtual lanes through the road map in response to the points trained to the trajectories, and generate a control signal that controls navigation of the vehicle through the portion of the road in response to the virtual lanes in the directed graph.

公开(公告)号：US20200310422A1

公开(公告)日：2020-10-01

申请号：US16365501

申请日：2019-03-26

Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC

Inventor： Rajan Bhattacharyya ,  Chong Ding ,  Vincent De Sapio ,  Michael J. Daily ,  Kyungnam Kim ,  Gavin D. Holland ,  Alexander S. Graber-Tilton ,  Kevin R. Martin

IPC: G05D1/00 ,  G05D1/02 ,  B60W50/00

Abstract: An autonomous vehicle, cognitive system for operating an autonomous vehicle and method of operating an autonomous vehicle. The cognitive system includes one or more hypothesizer modules, a hypothesis resolver, one or more decider modules, and a decision resolver. Data related to an agent is received at the cognitive system. The one or more hypothesizer modules create a plurality of hypotheses for a trajectory of the agent based on the received data. The hypothesis resolver selects a single hypothesis for the trajectory of the agent from the plurality of hypotheses based on a selection criteria. The one or more decider modules create a plurality of decisions for a trajectory of the autonomous vehicle based on the selected hypothesis for the agent. The decision resolver selects a trajectory for the autonomous vehicle from the plurality of decisions. The autonomous vehicle is operated based on the selected trajectory.

公开(公告)号：US20190332109A1

公开(公告)日：2019-10-31

申请号：US15964401

申请日：2018-04-27

Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC

Inventor： Soheil Kolouri ,  Cedrick G. Ngalande ,  Kyungnam Kim ,  Michael J. Daily

IPC: G05D1/00 ,  G05D1/02 ,  G06N3/04 ,  G06N3/08 ,  G06K9/00 ,  B60W10/20

Abstract: In various embodiments, methods, systems, and autonomous vehicles are provided. In one exemplary embodiment, a method is provided that includes obtaining first sensor inputs pertaining to one or more actors in proximity to an autonomous vehicle; obtaining second sensor inputs pertaining to operation of the autonomous vehicle; obtaining, via a processor, first neural network outputs via a first neural network, using the first sensor inputs; and obtaining, via the processor, second neural network outputs via a second neural network, using the first network outputs and the second sensor inputs, the second neural network outputs providing one or more recommended actions for controlling the autonomous vehicle.

公开(公告)号：US09844881B2

公开(公告)日：2017-12-19

申请号：US14746072

申请日：2015-06-22

Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC

Inventor： David W. Payton ,  Kyungnam Kim ,  Zhichao Chen ,  Ryan M. Uhlenbrock ,  Li Yang Ku

IPC: B25J9/16

CPC classification number: B25J9/1697

Abstract: A machine vision system for a controllable robotic device proximal to a workspace includes an image acquisition sensor arranged to periodically capture vision signal inputs each including an image of a field of view including the workspace. A controller operatively couples to the robotic device and includes a non-transitory memory component including an executable vision perception routine. The vision perception routine includes a focus loop control routine operative to dynamically track a focus object in the workspace and a background loop control routine operative to monitor a background of the workspace. The focus loop control routine executes simultaneously asynchronously in parallel with the background loop control routine to determine a combined resultant including the focus object and the background based upon the periodically captured vision signal inputs. The controller is operative to control the robotic device to manipulate the focus object based upon the focus loop control routine.

公开(公告)号：US20160368148A1

公开(公告)日：2016-12-22

申请号：US14746072

申请日：2015-06-22

Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC

Inventor： David W. Payton ,  Kyungnam Kim ,  Zhichao Chen ,  Ryan M. Uhlenbrock ,  Li Yang Ku

IPC: B25J9/16

CPC classification number: B25J9/1697

Abstract: A machine vision system for a controllable robotic device proximal to a workspace includes an image acquisition sensor arranged to periodically capture vision signal inputs each including an image of a field of view including the workspace. A controller operatively couples to the robotic device and includes a non-transitory memory component including an executable vision perception routine. The vision perception routine includes a focus loop control routine operative to dynamically track a focus object in the workspace and a background loop control routine operative to monitor a background of the workspace. The focus loop control routine executes simultaneously asynchronously in parallel with the background loop control routine to determine a combined resultant including the focus object and the background based upon the periodically captured vision signal inputs. The controller is operative to control the robotic device to manipulate the focus object based upon the focus loop control routine.

Abstract translation: 用于靠近工作空间的可控机器人设备的机器视觉系统包括图像采集传感器，其布置成周期性地捕获视觉信号输入，每个视觉信号输入包括包括工作空间的视野的图像。 控制器可操作地耦合到机器人设备并且包括包括可执行视觉感知程序的非暂时性存储器组件。 视觉感知例程包括可操作地动态地跟踪工作空间中的焦点对象的焦点循环控制例程和可操作以监视工作空间背景的背景环路控制程序。 焦点循环控制程序与背景环路控制程序并行异步执行，以基于周期性捕获的视觉信号输入来确定包括焦点对象和背景的组合结果。 控制器可操作以基于聚焦环控制程序来控制机器人装置来操纵焦点对象。

公开(公告)号：US09524426B2

公开(公告)日：2016-12-20

申请号：US14219109

申请日：2014-03-19

Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC

Inventor： Kyungnam Kim ,  Changsoo Jeong

IPC: G06K9/00 ,  H04N7/18 ,  G06K9/62 ,  G06K9/46

CPC classification number: G06K9/00369 ,  G06K9/00771 ,  G06K9/4671 ,  G06K9/6269 ,  H04N7/181

Abstract: A human monitoring system includes a plurality of cameras and a visual processor. The plurality of cameras are disposed about a workspace area, where each camera is configured to capture a video feed that includes a plurality of image frames, and the plurality of image frames are time-synchronized between the respective cameras. The visual processor is configured to receive the plurality of image frames from the plurality of vision-based imaging devices and detect the presence of a human from at least one of the plurality of image frames using pattern matching performed on an input image. The input image to the pattern matching is a sliding window portion of the image frame that is aligned with a rectified coordinate system such that a vertical axis in the workspace area is aligned with a vertical axis of the input image.

Abstract translation: 人类监控系统包括多个照相机和视觉处理器。 多个摄像机围绕工作区域设置，其中每个摄像机被配置为捕获包括多个图像帧的视频馈送，并且多个图像帧在各个相机之间是时间同步的。 视觉处理器被配置为从多个基于视觉的成像设备接收多个图像帧，并且使用在输入图像上执行的模式匹配从多个图像帧中的至少一个检测人的存在。 图案匹配的输入图像是与整流坐标系对准的图像帧的滑动窗口部分，使得工作区域中的垂直轴与输入图像的垂直轴对齐。

公开(公告)号：US09251598B2

公开(公告)日：2016-02-02

申请号：US14249386

申请日：2014-04-10

Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC

Inventor： James W. Wells ,  Kyungnam Kim ,  Swarup Medasani ,  Yuri Owechko

IPC: G06K9/00 ,  G06T7/20 ,  H04N7/18 ,  H04N5/225

CPC classification number: G06T7/2093 ,  G06K9/00771 ,  G06K2209/40 ,  G06T7/292 ,  G06T7/564 ,  G06T7/77 ,  G06T2207/30196 ,  G06T2207/30232 ,  H04N7/181

Abstract: A human monitoring system includes a plurality of cameras and a visual processor. The plurality of cameras are disposed about a workspace area, where each camera is configured to capture a video feed that includes a plurality of image frames, and the plurality of image frames are time-synchronized between the respective cameras. The visual processor is configured to receive the plurality of image frames from the plurality of vision-based imaging devices and determine an integrity score for each respective image frame. The processor may then isolate a foreground section from two or more of the views, determine a principle body axis for each respective foreground section, and determine a location point according to a weighted least squares function amongst the various principle body axes.

Abstract translation: 人类监控系统包括多个照相机和视觉处理器。 多个摄像机围绕工作区域设置，其中每个摄像机被配置为捕获包括多个图像帧的视频馈送，并且多个图像帧在各个相机之间是时间同步的。 可视处理器被配置为从多个基于视觉的成像设备接收多个图像帧，并且确定每个相应图像帧的完整性分数。 然后，处理器可以将前景部分与两个或更多个视图隔离，确定每个各个前景部分的主体轴线，并且根据各种原理体轴之间的加权最小二乘函数来确定位置点。