公开(公告)号：US12106518B2

公开(公告)日：2024-10-01

申请号：US17567371

申请日：2022-01-03

Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC

Inventor： Yao Hu ,  Xinyu Du ,  Binbin Li ,  Hao Yu ,  Wende Zhang

IPC: G06T7/73 ,  B60R11/04 ,  G06T7/80 ,  G06V20/58

CPC classification number: G06T7/80 ,  G06T7/73 ,  G06V20/58 ,  B60R11/04 ,  G06T2207/10016 ,  G06T2207/30252

Abstract: A system for on-vehicle camera alignment monitoring includes an on-vehicle camera in communication with a controller. The controller monitors vehicle operating parameters and camera signal parameters, and captures an image file from the on-vehicle camera. A first level analysis of the image file, the vehicle operating parameters, and the camera signal parameters is executed to detect dynamic conditions and image feature parameters that affect camera alignment. An error with one of the dynamic conditions or the image feature parameters that affects the camera alignment is detected. A second level analysis of the camera signal parameters is executed to identify a root cause indicating one of the dynamic conditions or the image feature parameters that affects the camera alignment based upon the error. A camera alignment-related fault is detected based upon the root cause, and vehicle operation is controlled based upon the camera alignment-related fault.

公开(公告)号：US11989893B2

公开(公告)日：2024-05-21

申请号：US17655888

申请日：2022-03-22

Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC

Inventor： Yousef Omar ,  Wende Zhang ,  Hao Yu

IPC: G06T7/37 ,  B60R1/22 ,  G01S17/87 ,  G01S17/89 ,  G06F18/2321 ,  G06F18/25 ,  G06V20/58

CPC classification number: G06T7/37 ,  B60R1/22 ,  G01S17/87 ,  G01S17/89 ,  G06F18/2321 ,  G06F18/251 ,  G06V20/58 ,  G06T2207/20081 ,  G06T2207/30252

Abstract: Systems and methods are provided for generating alignment parameters for processing data associated with a vehicle. In one embodiment, a method includes: receiving image data associated with an environment of the vehicle; receiving lidar data associated with the environment of the vehicle; processing, by a processor, the image data to determine data points associated with at least one pole identified within image data; processing, by the processor, the lidar data to determine data points associated with at least one pole identified within the lidar data; selectively storing the data points as data point pairs in a data buffer; iteratively processing, by the processor, the data point pairs with a plurality of perturbations to determine a transformation matrix; generating, by the processor, alignment data based on the transformation matrix; and processing future data based on the alignment parameters.

公开(公告)号：US11965978B2

公开(公告)日：2024-04-23

申请号：US17511959

申请日：2021-10-27

Applicant: GM Global Technology Operations LLC

Inventor： Kamran Ali ,  Wende Zhang ,  Fan Bai

IPC: G01S7/40 ,  B60W50/00 ,  B60W60/00

CPC classification number: G01S7/40 ,  B60W50/00 ,  B60W2050/0052 ,  B60W2050/0083 ,  B60W60/001 ,  B60W2420/408 ,  B60W2556/50

Abstract: A calibration pipeline for 6DoF alignment parameters for an autonomous vehicle includes an automated driving controller instructed to receive inertial measurement unit (IMU) poses and final radar poses and determine smoothened IMU poses from the IMU poses and smoothened final radar poses from the final radar poses. The automated driving controller aligns the smoothened IMU poses and the smoothened final radar poses with one another to create a plurality of radar-IMU A, B relative pose pairs. The automated riving controller determines a solution yielding a threshold number of inliers of further filtered radar-IMU A, B relative pose pairs, randomly samples the further filtered radar-IMU A, B relative pose pairs with replacements several times to determine a stream of filtered radar-IMU A, B relative pose pairs, and solves for a solution X for the stream of filtered radar-IMU A, B relative pose pairs.

公开(公告)号：US11853741B2

公开(公告)日：2023-12-26

申请号：US17647524

申请日：2022-01-10

Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC

Inventor： Wende Zhang ,  Esther Anderson ,  Mark T Gaylord ,  Robert A Dziurda

IPC: G06F8/65 ,  G06F8/71 ,  G06F18/214 ,  G06F18/2415 ,  G06T7/70 ,  G06V10/00 ,  G06V20/00

CPC classification number: G06F8/65 ,  G06F8/71 ,  G06F18/2148 ,  G06F18/2415 ,  G06T7/70 ,  G06V10/00 ,  G06V20/00 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/30252

Abstract: Vehicles and related systems and methods are provided for classifying detected objects in a location-dependent manner using localized models in a federated learning environment. A method involves obtaining sensor data for a detected object external to the vehicle from a sensor of a vehicle, obtaining location data associated with the detected object, obtaining a local classification model associated with an object type, assigning the object type to the detected object based on an output by the local classification model as a function of the sensor data and the location data using the local classification model, and initiating an action at the vehicle responsive to assigning the object type to the detected object.

公开(公告)号：US20230256975A1

公开(公告)日：2023-08-17

申请号：US17651406

申请日：2022-02-16

Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC

Inventor： Binbin Li ,  Xinyu Du ,  Yao Hu ,  Hao Yu ,  Wende Zhang

IPC: B60W40/10 ,  G06V20/58

CPC classification number: B60W40/10 ,  G06V20/58 ,  B60W2420/42 ,  B60W2552/53

Abstract: Systems and methods for a vehicle are provided. In one embodiment, a method includes: receiving image data defining a plurality of images associated with an environment of the vehicle; receiving vehicle data indicating a velocity of the vehicle, wherein the vehicle data is associated with the image data; determining, by a processor, feature points within at least one image based on the vehicle data and a three dimensional projection method; selecting, by the processor, a subset of the feature points as ground points; determining, by the processor, a ground plane based on the subset of feature points; determining, by the processor, a ground normal vector from the ground plane; determining, by the processor, a camera to ground alignment value based on the ground normal vector; and generating, by the processor, second image data based on the camera to ground alignment value.

公开(公告)号：US20230215026A1

公开(公告)日：2023-07-06

申请号：US17567365

申请日：2022-01-03

Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC

Inventor： Xinyu Du ,  Yao Hu ,  Binbin Li ,  Hao Yu ,  Wende Zhang

IPC: G06T7/33 ,  H04N5/247 ,  H04N5/262 ,  G06T7/246 ,  G06V20/56 ,  G06T7/55 ,  G06T7/80 ,  B60W10/04 ,  B60W10/18 ,  B60W10/20 ,  G01S17/89 ,  G01S17/931

CPC classification number: G06T7/33 ,  H04N5/247 ,  H04N5/2625 ,  G06T7/246 ,  G06V20/56 ,  G06T7/55 ,  G06T7/80 ,  B60W10/04 ,  B60W10/18 ,  B60W10/20 ,  G01S17/89 ,  G01S17/931 ,  G06T2207/30252 ,  G06T2207/20212 ,  G06T2207/10028 ,  B60W2420/42 ,  B60W2420/52 ,  B60W2552/35

Abstract: A vehicle control system including a spatial monitoring system includes on-vehicle cameras that capture images, from which are recovered a plurality of three-dimensional (3D) points. A left ground plane normal vector is determined for a left image, a center ground plane normal vector is determined for a front image, and a right ground plane normal vector is determined for a right image. A first angle difference between the left ground plane normal vector and the center ground plane normal vector is determined, and a second angle difference between the right ground plane normal vector and the center ground plane normal vector is determined. An uneven ground surface is determined based upon one of the first angle difference or the second angle difference, and an alignment compensation factor for the left camera or the right camera is determined. A bird's eye view image is determined based upon the alignment compensation factor.

公开(公告)号：US20230059883A1

公开(公告)日：2023-02-23

申请号：US17408758

申请日：2021-08-23

Applicant: GM Global Technology Operations LLC

Inventor： Yao Hu ,  Xinyu Du ,  Wende Zhang ,  Hao Yu

IPC: G01S17/931 ,  G06K9/46 ,  G06K9/62 ,  G01S17/89 ,  B60W40/06

Abstract: A system in a vehicle includes a lidar system to transmit incident light and receive reflections from one or more objects as a point cloud of points. The system also includes processing circuitry to identify feature points among the points of the point cloud, the feature points being horizontal feature points reflected from a horizontal surface or vertical feature points reflected from a vertical surface. The processing circuitry processes the point cloud by obtaining a normal vector corresponding to each of the points of the point cloud. The normal vector includes a first component associated with a first dimension, a second component associated with a second dimension, and a third component associated with a third dimension.

公开(公告)号：US11428541B1

公开(公告)日：2022-08-30

申请号：US17329613

申请日：2021-05-25

Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC

Inventor： Mohsen Khalili ,  Hao Yu ,  Wende Zhang

IPC: G01C25/00 ,  G06T7/70

Abstract: A calibration system for a camera and an inertial sensor, respectively rigidly affixed to a vehicle, includes a controller having a processor and tangible, non-transitory memory on which instructions are recorded. The camera is adapted to capture one or more images and the inertial sensor is adapted to obtain motion data, including an angular velocity vector of the vehicle. The controller is configured to receive respective data from the camera and the inertial sensor. The controller is configured to perform adaptive feature selection on the one or more images, including extracting one or more regions of interest respectively from the images and calculating a respective optimal quality level for each of the regions of interest. The camera is aligned to the inertial sensor based in part on the adaptive feature selection and the respective data from the inertial sensor.

公开(公告)号：US11288553B1

公开(公告)日：2022-03-29

申请号：US17072095

申请日：2020-10-16

Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC

Inventor： Mohannad Murad ,  Wende Zhang ,  Kevin K. Hoang

IPC: G06K9/62 ,  G06K9/00

Abstract: A system and method are provided for aiding an operator in operating a vehicle. In one embodiment, a system includes a sensor system configured to generate sensor data sensed from an environment of the vehicle. The system further includes a control module configured to, by a processor, generate a bowl view image of the environment based on the sensor data, identify at least one of a double object and a missing object in the bowl view image, generate a second bowl view image of the environment by re-stitching the sensor data based on the at least one of double object and missing object, and generate display data based on the second bowl view image.

公开(公告)号：US20210335007A1

公开(公告)日：2021-10-28

申请号：US16859436

申请日：2020-04-27

Applicant: GM Global Technology Operations LLC

Inventor： Wende Zhang ,  Fred W. Huntzicker ,  Hao Yu

IPC: G06T7/80 ,  G06T7/73

Abstract: A system and method to perform automatic alignment of an optical sensor of a vehicle involve disposing two or more active targets at known locations in an alignment station. Each of the two or more active targets have at least two visibly different states. The method includes coding a change among the at least two visibly different states for the two or more active targets. Images obtained by the optical sensor of the two or more active targets are processed to identify features and perform the alignment of the optical sensor with another sensor of the vehicle or with the vehicle.