公开(公告)号：US20250148639A1

公开(公告)日：2025-05-08

申请号：US18500395

申请日：2023-11-02

Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC

Inventor： Binbin Li ,  Xinyu Du ,  Yao Hu ,  Hao Yu ,  Wende Zhang

IPC: G06T7/73 ,  G06V10/44 ,  G06V10/762 ,  G06V20/56

Abstract: A method of performing a camera to ground alignment for a camera system on a vehicle. The method includes determining if enabling conditions have occurred and estimating a location of a vanishing point in a source image. Ground lines are selected based on a source image. A lane line detection is performed based on clustering of the ground lines to determine lane lines in the source image. At least one of pitch, yaw, or roll of the vehicle are estimated from the source image. A cost function based on estimates of pitch, yaw, and roll is minimized to obtain an optimal pitch value, an optimal yaw value, an optimal roll value, and lane lines from the source image. A sliding window-based refinement is performed on source images. Alignment results are broadcast to a downstream application or it is determined if the camera system on the vehicle is misaligned.

公开(公告)号：US12130390B2

公开(公告)日：2024-10-29

申请号：US17569948

申请日：2022-01-06

Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC

Inventor： Xinyu Du ,  Yao Hu ,  Wende Zhang ,  Hao Yu

IPC: G01S7/497 ,  B60W60/00 ,  G01S17/86 ,  G01S17/89 ,  G01S17/931

CPC classification number: G01S7/4972 ,  B60W60/00 ,  G01S17/86 ,  G01S17/89 ,  G01S17/931 ,  B60W2420/408

Abstract: A LIDAR-to-vehicle alignment system includes a memory and alignment and autonomous driving modules. The memory stores points of data provided based on an output of one or more LIDAR sensors and localization data. The alignment module performs an alignment process including: based on the localization data; determining whether a host vehicle is turning; in response to the host vehicle turning; selecting a portion of the points of data; aggregating the selected portion to provide aggregated data; selecting targets based on the aggregated data; and based on the selected targets, iteratively reducing a loss value of a loss function to provide a resultant LIDAR-to-vehicle transformation matrix. The autonomous driving module: based on the resultant LIDAR-to-vehicle transformation matrix, converts at least the selected portion to at least one of vehicle coordinates or world coordinates to provide resultant data; and performs one or more autonomous driving operations based on the resultant data.

公开(公告)号：US12094169B2

公开(公告)日：2024-09-17

申请号：US17651407

申请日：2022-02-16

Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC

Inventor： Binbin Li ,  Xinyu Du ,  Yao Hu ,  Hao Yu ,  Wende Zhang

IPC: G06K9/00 ,  G06T7/73 ,  G06T7/80 ,  G06V10/22 ,  G06V10/40

CPC classification number: G06T7/80 ,  G06T7/73 ,  G06V10/22 ,  G06V10/40 ,  G06T2207/30244 ,  G06T2207/30252

Abstract: Systems and methods for a vehicle are provided. In one embodiment, a method includes: receiving image data defining a plurality of images associated with an environment of the vehicle; determining, by a processor, feature points within at least one image of the plurality of images; selecting, by the processor, a subset of the feature points as ground points based on a fixed two dimensional image road mask and a three dimensional region; determining, by the processor, a ground plane based on the subset of feature points; determining, by the processor, a ground normal vector from the ground plane; determining, by the processor, a camera to ground alignment value based on the ground normal vector; and generating, by the processor, second image data based on the camera to ground alignment value.

公开(公告)号：US12086996B2

公开(公告)日：2024-09-10

申请号：US17567365

申请日：2022-01-03

Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC

Inventor： Xinyu Du ,  Yao Hu ,  Binbin Li ,  Hao Yu ,  Wende Zhang

IPC: G06T7/00 ,  B60W10/04 ,  B60W10/18 ,  B60W10/20 ,  G01S17/89 ,  G01S17/931 ,  G06T7/246 ,  G06T7/33 ,  G06T7/55 ,  G06T7/80 ,  G06V20/56 ,  H04N5/262 ,  H04N23/90

CPC classification number: G06T7/33 ,  B60W10/04 ,  B60W10/18 ,  B60W10/20 ,  G01S17/89 ,  G01S17/931 ,  G06T7/246 ,  G06T7/55 ,  G06T7/80 ,  G06V20/56 ,  H04N5/2625 ,  H04N23/90 ,  B60W2420/403 ,  B60W2420/408 ,  B60W2552/35 ,  G06T2207/10028 ,  G06T2207/20212 ,  G06T2207/30252

Abstract: A vehicle control system including a spatial monitoring system includes on-vehicle cameras that capture images, from which are recovered a plurality of three-dimensional points. A left ground plane normal vector is determined for a left image, a center ground plane normal vector is determined for a front image, and a right ground plane normal vector is determined for a right image. A first angle difference between the left ground plane normal vector and the center ground plane normal vector is determined, and a second angle difference between the right ground plane normal vector and the center ground plane normal vector is determined. An uneven ground surface is determined based upon one of the first angle difference or the second angle difference, and an alignment compensation factor for the left camera or the right camera is determined. A bird's eye view image is determined based upon the alignment compensation factor.

公开(公告)号：US20240095954A1

公开(公告)日：2024-03-21

申请号：US17947550

申请日：2022-09-19

Applicant: GM Global Technology Operations LLC

Inventor： Yao Hu ,  Hao Yu ,  Guanlun He ,  Xinyu Du ,  Binbin Li

IPC: G06T7/80 ,  G06T5/00 ,  G06T15/10

CPC classification number: G06T7/80 ,  G06T5/006 ,  G06T15/10 ,  G06T2207/30268

Abstract: A vehicle includes a system performing a method of operating a camera for the vehicle. The camera is configured to capture an image. The processor is configured to query the camera to obtain an intrinsic parameter of the camera, estimate a unified parameter based on the intrinsic parameter of the camera, and perform at least one application on the image using the unified parameter.

公开(公告)号：US11904899B2

公开(公告)日：2024-02-20

申请号：US17469409

申请日：2021-09-08

Applicant: GM Global Technology Operations LLC

Inventor： Rasoul Salehi ,  Arun Adiththan ,  Yao Hu ,  Yilu Zhang

IPC: B60W50/023 ,  B60W60/00 ,  B60W50/029 ,  B60W50/02

CPC classification number: B60W60/00186 ,  B60W50/023 ,  B60W50/0205 ,  B60W50/029 ,  B60W50/0225 ,  B60W2050/0215 ,  B60W2050/0292 ,  B60W2420/42 ,  B60W2420/52 ,  B60W2552/05 ,  B60W2556/50 ,  B60W2556/60 ,  B60W2720/10

Abstract: An autonomous driving system for an autonomous vehicle includes a plurality of on-board autonomous sensors that sense data related to operation of the autonomous vehicle and a surrounding environment and an automated driving controller in electronic communication with the plurality of on-board autonomous sensors. The automated driving controller is instructed to receive an indication one or more of the plurality of on-board autonomous sensors are non-functional and a secondary autonomous sensor system including one or more replacement sensors are installed. The automated driving controller is instructed to verify the secondary autonomous sensor system based on a security check and perform a redundancy check between the one or more replacement sensors and the plurality of on-board autonomous sensors. In response to determining the one or more replacement sensors are valid based on the redundancy check, the automated driving controller operates the autonomous vehicle in a limp home mode.

公开(公告)号：US11851073B2

公开(公告)日：2023-12-26

申请号：US17557156

申请日：2021-12-21

Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC

Inventor： Wen-Chiao Lin ,  Zhongyu Yang ,  Yao Hu ,  Xinyu Du ,  Yilu Zhang

IPC: B60W50/00 ,  B60W50/02 ,  B60W30/12 ,  B60W50/029

CPC classification number: B60W50/02 ,  B60W30/12 ,  B60W50/0225 ,  B60W2050/0072 ,  B60W2050/021 ,  B60W2050/0215 ,  B60W2050/0297 ,  B60W2552/53 ,  B60W2555/20

Abstract: A system for a vehicle includes a plurality of sensors onboard the vehicle and a controller. A first sensor of the plurality of sensors is configured to detect lane markings on a roadway. The controller is configured to store data from the plurality of sensors. In response to receiving an indication indicating a misdetection of lane markings on the roadway based on data received from the first sensor, the controller is configured to execute in parallel a plurality of procedures configured to detect a plurality of causes for the misdetection of lane markings, respectively, based on the stored data; isolate one of the causes as a root cause for the misdetection of lane markings; and provide a response for mitigating the misdetection of lane markings on the roadway based on the root cause for the misdetection of lane markings.

公开(公告)号：US11801870B2

公开(公告)日：2023-10-31

申请号：US17471323

申请日：2021-09-10

Applicant: GM Global Technology Operations LLC

Inventor： Rasoul Salehi ,  Yao Hu ,  Arun Adiththan ,  Yilu Zhang

IPC: B60W60/00 ,  G07C5/08 ,  B60W50/00 ,  B60W10/18 ,  B60W10/20 ,  B60W10/04 ,  B60W10/22 ,  H04W4/46 ,  B60W50/035

CPC classification number: B60W60/0025 ,  B60W10/04 ,  B60W10/18 ,  B60W10/20 ,  B60W10/22 ,  B60W50/0097 ,  B60W50/035 ,  G07C5/0816 ,  H04W4/46 ,  B60W2420/42 ,  B60W2420/52 ,  B60W2520/06 ,  B60W2520/10 ,  B60W2520/12 ,  B60W2540/10 ,  B60W2540/12 ,  B60W2540/18 ,  B60W2556/60 ,  B60W2556/65 ,  B60W2710/18 ,  B60W2710/20 ,  B60W2710/22

Abstract: An autonomous driving system for an autonomous vehicle includes an automated driving controller wirelessly connected to a towing taxi. The automated driving controller determines the autonomous driving system is non-functional. In response to determining the autonomous driving system is non-functional, the automated driving controller generates a notification indicating the autonomous driving system is non-functional. The automated driving controller receives, from the towing taxi, a current data string including a data point corresponding to a current point in time in combination with a predicted data point for each of one or more predicted points of time in the future. The current data string is compared with a previous data string recorded at a previous point in time. In response to determining the current data string matches the previous data string, the automated driving controller determines one or more driving maneuvers for the autonomous vehicle based on the current data string.

公开(公告)号：US20230132179A1

公开(公告)日：2023-04-27

申请号：US17452299

申请日：2021-10-26

Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC

Inventor： Rasoul Salehi ,  Arun Adiththan ,  Yilu Zhang ,  Yao Hu

IPC: B60W60/00 ,  G05D1/10 ,  G07C5/00

Abstract: Methods and systems for a remote transportation system including a first autonomous vehicle, at least one second autonomous vehicle and a remote transportation server are provided. The at least one second autonomous vehicle includes non-transitory computer readable media and one or more processors configured by programming instructions on the non-transitory computer readable media to: receive a request for tow service from the remote transportation server, wherein the request includes a location of the first autonomous vehicle; locate and identify the first autonomous vehicle based on the request; create a communicate link between the first autonomous vehicle and the second autonomous vehicle; select at least one of a centralized towing method and a projection-based towing method based on the request; and perform autonomous towing of the first autonomous vehicle based on the selected of the at least one of the centralized towing method and the projection-based towing method.

公开(公告)号：US10981577B1

公开(公告)日：2021-04-20

申请号：US16720096

申请日：2019-12-19

Applicant: GM GLOBAL TECHNOLOGY OPERATIONS LLC

Inventor： Yao Hu

IPC: B60Q1/00 ,  B60W50/02 ,  G06K9/00 ,  G06F40/30 ,  G06N3/04 ,  B60W50/14

Abstract: A perception module receives frames of data from sensors onboard a vehicle and identifies objects surrounding the vehicle based on the data. A fault detection module determines semantic information associated with the data, generates a score for each of the identified objects based on a correlation of each of the identified objects with the semantic information, and generates a score for the frames based on the scores for the identified objects. The fault detection module determines that the perception module: is operating normally in response to the score for the frames being less than a first threshold, has a fault in response to the score for the frames being greater than a second threshold, and needs inspection in response to the score being between the first and second thresholds.