公开(公告)号：US11954916B2

公开(公告)日：2024-04-09

申请号：US17665895

申请日：2022-02-07

Applicant: GM Global Technology Operations LLC

Inventor： Wei Tong ,  Siddhartha Gupta ,  Jacob Alan Bond ,  Zhuoning Yuan

IPC: G06V20/00 ,  B60W40/04 ,  B60W60/00 ,  G06V10/40 ,  G06V10/74 ,  G06V10/764 ,  G06V10/82 ,  G06V20/58

CPC classification number: G06V20/58 ,  B60W40/04 ,  B60W60/001 ,  G06V10/40 ,  G06V10/761 ,  G06V10/764 ,  G06V10/82 ,  B60W2420/403 ,  B60W2554/4026 ,  B60W2554/4029

Abstract: An automated driving system includes an object detection system. A neural network image encoder generates image embeddings associated with an image including an object. A neural network text encoder generates concept embeddings associated with each of a plurality of concepts. Each of the plurality of concepts is associated with one of at least two object classes. A confidence score module generates a confidence score for each of the plurality of concepts based on the image embeddings and the concept embeddings associated with the concept. An object class prediction module generates a predicted object class of the object based on an association between a set of concepts of the plurality of concepts having at least two of the highest values of the generated confidence scores and the one of the at least two object classes associated with a majority of the set of concepts.

公开(公告)号：US20240071228A1

公开(公告)日：2024-02-29

申请号：US17822897

申请日：2022-08-29

Applicant: GM Global Technology Operations LLC

Inventor： Jacob Alan Bond ,  Joseph F. Szczerba ,  John P. Weiss

IPC: G08G1/16

CPC classification number: G08G1/165 ,  G06V20/588

Abstract: A system for determining when additional information is needed by a passenger with color deficient vision within a vehicle includes a plurality of perception sensors within the vehicle adapted to collect data related to a physical identity of a primary object and to communicate the data related to the physical identity of the primary object, via a communication bus, to a data processor within the vehicle, the data processor adapted to determine that the physical identity of the primary object may be unclear to a passenger with color deficient vision based on the data related to the physical identity of the primary object, and that assistance should be provided to the passenger with color deficient vision to aid the passenger with color deficient vision in identifying the primary object.

公开(公告)号：US20230282210A1

公开(公告)日：2023-09-07

申请号：US17685716

申请日：2022-03-03

Applicant: GM Global Technology Operations LLC

Inventor： Jacob Alan Bond ,  Hannah Elizabeth Wagner ,  Joseph F. Szczerba ,  Alan D. Hejl

IPC: G10L15/22 ,  G10L15/16 ,  G10L15/25 ,  G06T7/20 ,  G06K9/62 ,  B60W40/08 ,  G06N3/08

CPC classification number: G10L15/22 ,  B60W40/08 ,  G06K9/6289 ,  G06N3/08 ,  G06T7/20 ,  G10L15/16 ,  G10L15/25 ,  B60W2040/089 ,  B60W2420/42 ,  B60W2420/54 ,  B60W2540/21 ,  B60W2540/221 ,  G06T2207/30201 ,  G10L2015/223

Abstract: A speech recognition method includes receiving audible data and user data. The audible data includes information about an utterance by the user. The user data includes information about movements by the user. The method further includes fusing the audible data and the user data to obtain fused data and determining at least one spoken word of the utterance based on the fused data.

公开(公告)号：US20230252795A1

公开(公告)日：2023-08-10

申请号：US17665895

申请日：2022-02-07

Applicant: GM Global Technology Operations LLC

Inventor： Wei Tong ,  Siddhartha Gupta ,  Jacob Alan Bond ,  Zhuoning Yuan

IPC: G06V20/58 ,  G06V10/82 ,  G06V10/764 ,  G06V10/40 ,  G06V10/74 ,  B60W60/00 ,  B60W40/04

CPC classification number: G06V20/58 ,  G06V10/82 ,  G06V10/764 ,  G06V10/40 ,  G06V10/761 ,  B60W60/001 ,  B60W40/04 ,  B60W2554/4026 ,  B60W2554/4029 ,  B60W2420/42

Abstract: An automated driving system includes an object detection system. A neural network image encoder generates image embeddings associated with an image including an object. A neural network text encoder generates concept embeddings associated with each of a plurality of concepts. Each of the plurality of concepts is associated with one of at least two object classes. A confidence score module generates a confidence score for each of the plurality of concepts based on the image embeddings and the concept embeddings associated with the concept. An object class prediction module generates a predicted object class of the object based on an association between a set of concepts of the plurality of concepts having at least two of the highest values of the generated confidence scores and the one of the at least two object classes associated with a majority of the set of concepts.

公开(公告)号：US20230185919A1

公开(公告)日：2023-06-15

申请号：US17551763

申请日：2021-12-15

Applicant: GM Global Technology Operations LLC

Inventor： Jacob Alan Bond

IPC: G06F21/57 ,  H04L9/00 ,  G06F21/60 ,  G06N3/10

CPC classification number: G06F21/57 ,  H04L9/008 ,  G06F21/602 ,  G06N3/10

Abstract: A computer is provided for a system of a motor vehicle, with the system including one or more perception sensors generating sensor data and one or more output devices performing an action, in response to the output device receiving an output signal. The computer includes one or more processors coupled to the perception sensor and the output device, with the processor having a Rich Execution Environment (REE) and a Trusted Execution Environment (TEE). The computer further includes a non-transitory computer readable storage medium (CRM) including instructions such that the at least one processor is programmed to: receive the sensor data from the perception sensor; transfer the sensor data from the REE to the TEE; homomorphically encrypt the sensor data in the TEE; and calculate the encrypted output based on the encrypted sensor data in the REE and a plurality of weights of a homomorphically encrypted neural network in the REE.

公开(公告)号：US20230162039A1

公开(公告)日：2023-05-25

申请号：US17535129

申请日：2021-11-24

Applicant: GM Global Technology Operations LLC

Inventor： Siddhartha Gupta ,  Jacob Alan Bond ,  Wei Tong ,  Upali P. Mudalige

IPC: G06N3/08 ,  G06V10/75 ,  G06V10/82

CPC classification number: G06N3/084 ,  G06V10/751 ,  G06V10/82

Abstract: A system comprises a computer including a processor and a memory. The memory includes instructions such that the processor is programmed to: receive, at a selective dropout layer of a neural network, a plurality of adversarial image features and a plurality of natural image features, select one or more nodes within the selective dropout layer to deactivate based on a comparison of the plurality of adversarial image features with the plurality of natural image features, and deactivate the selected one or more nodes.

公开(公告)号：US11645038B1

公开(公告)日：2023-05-09

申请号：US17828121

申请日：2022-05-31

Applicant: GM Global Technology Operations LLC

Inventor： Jacob Alan Bond ,  Joseph F. Szczerba ,  Hannah Elizabeth Wagner ,  Kai-Han Chang ,  Thomas A. Seder

IPC: G06F3/16 ,  G06F3/01

CPC classification number: G06F3/167 ,  G06F3/011

Abstract: An augmented reality head up display system includes an audio microphone array, a driver position tracking device, and an augmented reality head up display. The augmented reality head up display system also includes a controller in electronic communication with the audio microphone array, the driver position tracking device, and the augmented reality head up display, where the controller is programmed to: receive audio data from the audio microphone array corresponding to audio detected by the audio microphone array, classify the audio data as corresponding to an audio event occurring in an environment surrounding the vehicle, and display a graphic corresponding to the audio event on the windscreen of the vehicle using the augmented reality head up display and input from the driver position tracking device.

公开(公告)号：US11492005B1

公开(公告)日：2022-11-08

申请号：US17410189

申请日：2021-08-24

Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC

Inventor： Jacob Alan Bond

IPC: H04N5/44 ,  B60W50/14 ,  B60W40/08 ,  G10L21/10 ,  B60W40/02 ,  G10L21/06

Abstract: Methods and systems for controlling an occupant output system associated with a vehicle are provided. The methods and systems receive vehicle or occupant context data from a source of vehicle context data, generate occupant message data based on the vehicle or occupant context data and determine if an occupant associated with the occupant output system is vision or hearing impaired. When the occupant is determined to be vision or hearing impaired, the methods and systems decide on an output modality to assist the occupant, and generate an output for the occupant on the output device, and in the output modality, based on the occupant message data.