公开(公告)号：US12243328B2

公开(公告)日：2025-03-04

申请号：US17820317

申请日：2022-08-17

Applicant: GM Global Technology Operations LLC

Inventor： Jiejun Xu ,  Kenji Yamada ,  Michael J. Daily ,  Alireza Esna Ashari Esfahani ,  Hyukseong Kwon ,  Darren Michael Chan ,  Alan Perry ,  Joshua Lampkins

IPC: G06V20/58 ,  G06N3/045 ,  G06V10/82 ,  G06V20/62 ,  G06V30/262

Abstract: A road sign interpretation system includes a front-facing camera mounted on or in a vehicle collecting image data of multiple road signs. A first convolutional neural network (CNN) receives the image data from the front-facing camera and yields a set of sign predictions including one or more sign text instances. A second CNN defining a text extractor receives the image data from the front-facing camera and extracts text candidates including the multiple sign text instances. Sign and sign data localization is provided in the second CNN to compute a text order from the multiple sign text instances. A sign text synthesizer module receives individual sign text instances from the first CNN and individual ones of the sign text instances in digitized forms from an optical character recognizer (OCR). A semantic encoding and interpretation module receives the sign text instances and identifies semantics of the multiple road signs.

公开(公告)号：US20240371177A1

公开(公告)日：2024-11-07

申请号：US18311294

申请日：2023-05-03

Applicant: GM Global Technology Operations LLC

Inventor： Darren Michael Chan ,  Jiejun Xu ,  Alireza Esna Ashari Esfahani

IPC: G06V20/58 ,  G01S19/42 ,  G06T7/12 ,  G06T7/70 ,  G06V10/20 ,  G06V10/25

Abstract: A system for determining a relevance of a traffic sign for a vehicle includes at least one vehicle camera configured to provide a view of an environment surrounding the vehicle and a vehicle controller in electrical communication with the at least one vehicle camera. The vehicle controller is programmed to capture an image using the at least one vehicle camera. The vehicle controller is further programmed to identify the traffic sign in the image. The vehicle controller is further programmed to determine a pan angle and a tilt angle of the traffic sign based at least in part on the image. The vehicle controller is further programmed to determine the relevance of the traffic sign based at least in part on the pan angle and the tilt angle of the traffic sign.

公开(公告)号：US20240062555A1

公开(公告)日：2024-02-22

申请号：US17820317

申请日：2022-08-17

Applicant: GM Global Technology Operations LLC

Inventor： Jiejun Xu ,  Kenji Yamada ,  Michael J. Daily ,  Alireza Esna Ashari Esfahani ,  Hyukseong Kwon ,  Darren Michael Chan ,  Alan Perry ,  Joshua Lampkins

IPC: G06V20/58 ,  G06V20/62 ,  G06V10/82 ,  G06V30/262 ,  G06N3/04

CPC classification number: G06V20/582 ,  G06V20/63 ,  G06V10/82 ,  G06V30/274 ,  G06N3/0454

Abstract: A road sign interpretation system includes a front-facing camera mounted on or in a vehicle collecting image data of multiple road signs. A first convolutional neural network (CNN) receives the image data from the front-facing camera and yields a set of sign predictions including one or more sign text instances. A second CNN defining a text extractor receives the image data from the front-facing camera and extracts text candidates including the multiple sign text instances. Sign and sign data localization is provided in the second CNN to compute a text order from the multiple sign text instances. A sign text synthesizer module receives individual sign text instances from the first CNN and individual ones of the sign text instances in digitized forms from an optical character recognizer (OCR). A semantic encoding and interpretation module receives the sign text instances and identifies semantics of the multiple road signs.

公开(公告)号：US09875427B2

公开(公告)日：2018-01-23

申请号：US14811062

申请日：2015-07-28

Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC

Inventor： Swarup Medasani ,  Jason Meltzer ,  Jiejun Xu ,  Zhichao Chen ,  Rashmi N. Sundareswara ,  David W. Payton ,  Ryan M. Uhlenbrock ,  Leandro G. Barajas ,  Kyungnam Kim

IPC: G06K9/00 ,  G06K9/62 ,  G01B11/00 ,  G06K9/46 ,  G06T3/00 ,  G06T7/20 ,  G06T7/73

CPC classification number: G06K9/6218 ,  G01B11/002 ,  G06K9/00 ,  G06K9/00214 ,  G06K9/4671 ,  G06K9/6203 ,  G06K9/6215 ,  G06K9/6256 ,  G06K2009/4666 ,  G06T3/00 ,  G06T7/20 ,  G06T7/74 ,  G06T2207/10028 ,  G06T2207/20016

Abstract: A method for localizing and estimating a pose of a known object in a field of view of a vision system is described, and includes developing a processor-based model of the known object, capturing a bitmap image file including an image of the field of view including the known object, extracting features from the bitmap image file, matching the extracted features with features associated with the model of the known object, localizing an object in the bitmap image file based upon the extracted features, clustering the extracted features of the localized object, merging the clustered extracted features, detecting the known object in the field of view based upon a comparison of the merged clustered extracted features and the processor-based model of the known object, and estimating a pose of the detected known object in the field of view based upon the detecting of the known object.

公开(公告)号：US20170032220A1

公开(公告)日：2017-02-02

申请号：US14811062

申请日：2015-07-28

Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC

Inventor： Swarup Medasani ,  Jason Meltzer ,  Jiejun Xu ,  Zhichao Chen ,  Rashmi N. Sundareswara ,  David W. Payton ,  Ryan M. Uhlenbrock ,  Leandro G. Barajas ,  Kyungnam Kim

IPC: G06K9/62 ,  G06T7/00 ,  G06T7/20 ,  G06K9/46 ,  G01B11/00 ,  G06T3/00

CPC classification number: G06K9/6218 ,  G01B11/002 ,  G06K9/00 ,  G06K9/00214 ,  G06K9/4671 ,  G06K9/6203 ,  G06K9/6215 ,  G06K9/6256 ,  G06K2009/4666 ,  G06T3/00 ,  G06T7/20 ,  G06T7/74 ,  G06T2207/10028 ,  G06T2207/20016

Abstract: A method for localizing and estimating a pose of a known object in a field of view of a vision system is described, and includes developing a processor-based model of the known object, capturing a bitmap image file including an image of the field of view including the known object, extracting features from the bitmap image file, matching the extracted features with features associated with the model of the known object, localizing an object in the bitmap image file based upon the extracted features, clustering the extracted features of the localized object, merging the clustered extracted features, detecting the known object in the field of view based upon a comparison of the merged clustered extracted features and the processor-based model of the known object, and estimating a pose of the detected known object in the field of view based upon the detecting of the known object.

Abstract translation: 描述了一种用于在视觉系统的视野中定位和估计已知对象的姿态的方法，并且包括开发已知对象的基于处理器的模型，捕获包括视场图像的位图图像文件 包括已知对象，从位图图像文件中提取特征，将所提取的特征与已知对象的模型相关联的特征进行匹配，基于所提取的特征将位图图像文件中的对象定位，将所提取的特征进行聚类 ，合并聚集提取的特征，基于所合并的聚类提取特征与已知对象的基于处理器的模型的比较，在视野中检测已知对象，以及估计已知对象的姿势 基于已知对象的检测来查看。

公开(公告)号：US20240311566A1

公开(公告)日：2024-09-19

申请号：US18185753

申请日：2023-03-17

Applicant: GM Global Technology Operations LLC

Inventor： Sasha Strelnikoff ,  Jiejun Xu ,  Alireza Esna Ashari Esfahani

IPC: G06F40/289 ,  B60K35/00 ,  G06F40/205 ,  G06V20/58 ,  G06V30/262

CPC classification number: G06F40/289 ,  B60K35/00 ,  G06F40/205 ,  G06V20/582 ,  G06V30/262 ,  B60K35/10 ,  B60K35/22 ,  B60K2360/148

Abstract: A method for language processing for a vehicle includes receiving an input text. The input text includes a plurality of words. The method also includes determining a rule-based action representation of the input text, parsing the input text to produce a parsed text. The method also includes determining a model-based action representation of the parsed text. The method also includes determining a final action representation of the input text based at least in part on the rule-based action representation and the model-based action representation.