公开(公告)号：US20220197284A1

公开(公告)日：2022-06-23

申请号：US17692706

申请日：2022-03-11

Applicant: NVIDIA Corporation

Inventor： Nikolai Smolyanskiy ,  Alexey Kamenev ,  Jeffrey David Smith ,  Stanley Thomas Birchfield

IPC: G05D1/00 ,  G05D1/02 ,  B62D6/00 ,  G06N3/08 ,  G06N7/00 ,  B62D15/02 ,  G06K9/62 ,  G05D1/10 ,  G06K9/00 ,  G06N3/04 ,  G06V10/94 ,  G06V20/56

Abstract: A method, computer readable medium, and system are disclosed for performing autonomous path navigation using deep neural networks. The method includes the steps of receiving image data at a deep neural network (DNN), determining, by the DNN, both an orientation of a vehicle with respect to a path and a lateral position of the vehicle with respect to the path, utilizing the image data, and controlling a location of the vehicle, utilizing the orientation of the vehicle with respect to the path and the lateral position of the vehicle with respect to the path.

公开(公告)号：US20210326678A1

公开(公告)日：2021-10-21

申请号：US17356140

申请日：2021-06-23

Applicant: NVIDIA Corporation

Inventor： Nikolai Smolyanskiy ,  Alexey Kamenev ,  Stan Birchfield

IPC: G06N3/04 ,  G06T7/593 ,  G06N3/08 ,  G06T1/20 ,  G06K9/62 ,  G06N3/063 ,  G01S17/86 ,  G01S17/89

Abstract: Various examples of the present disclosure include a stereoscopic deep neural network (DNN) that produces accurate and reliable results in real-time. Both LIDAR data (supervised training) and photometric error (unsupervised training) may be used to train the DNN in a semi-supervised manner. The stereoscopic DNN may use an exponential linear unit (ELU) activation function to increase processing speeds, as well as a machine learned argmax function that may include a plurality of convolutional layers having trainable parameters to account for context. The stereoscopic DNN may further include layers having an encoder/decoder architecture, where the encoder portion of the layers may include a combination of three-dimensional convolutional layers followed by two-dimensional convolutional layers.

公开(公告)号：US20230169321A1

公开(公告)日：2023-06-01

申请号：US18160694

申请日：2023-01-27

Applicant: NVIDIA Corporation

Inventor： Nikolai Smolyanskiy ,  Alexey Kamenev ,  Stan Birchfield

IPC: G06N3/063 ,  G06T7/593 ,  G06N3/084 ,  G06N3/088 ,  G06T1/20 ,  G01S17/86 ,  G01S17/89 ,  G06F18/22 ,  G06N3/045 ,  G06N3/048

CPC classification number: G06N3/063 ,  G06T7/593 ,  G06N3/084 ,  G06N3/088 ,  G06T1/20 ,  G01S17/86 ,  G01S17/89 ,  G06F18/22 ,  G06N3/045 ,  G06N3/048 ,  G06T2207/10052 ,  G06T2207/20084 ,  G06T2207/10012

Abstract: Various examples of the present disclosure include a stereoscopic deep neural network (DNN) that produces accurate and reliable results in real-time. Both LIDAR data (supervised training) and photometric error (unsupervised training) may be used to train the DNN in a semi-supervised manner. The stereoscopic DNN may use an exponential linear unit (ELU) activation function to increase processing speeds, as well as a machine learned argmax function that may include a plurality of convolutional layers having trainable parameters to account for context. The stereoscopic DNN may further include layers having an encoder/decoder architecture, where the encoder portion of the layers may include a combination of three-dimensional convolutional layers followed by two-dimensional convolutional layers.

公开(公告)号：US11604967B2

公开(公告)日：2023-03-14

申请号：US17356140

申请日：2021-06-23

Applicant: NVIDIA Corporation

Inventor： Nikolai Smolyanskiy ,  Alexey Kamenev ,  Stan Birchfield

IPC: G01S17/88 ,  G01S17/894 ,  G06N3/02 ,  G06N3/084 ,  G06T7/50 ,  G06T7/80 ,  G06N3/04 ,  G06T7/593 ,  G06N3/088 ,  G06T1/20 ,  G06K9/62 ,  G06N3/063 ,  G01S17/86 ,  G01S17/89

Abstract: Various examples of the present disclosure include a stereoscopic deep neural network (DNN) that produces accurate and reliable results in real-time. Both LIDAR data (supervised training) and photometric error (unsupervised training) may be used to train the DNN in a semi-supervised manner. The stereoscopic DNN may use an exponential linear unit (ELU) activation function to increase processing speeds, as well as a machine learned argmax function that may include a plurality of convolutional layers having trainable parameters to account for context. The stereoscopic DNN may further include layers having an encoder/decoder architecture, where the encoder portion of the layers may include a combination of three-dimensional convolutional layers followed by two-dimensional convolutional layers.

公开(公告)号：US20210026355A1

公开(公告)日：2021-01-28

申请号：US16938706

申请日：2020-07-24

Applicant: NVIDIA Corporation

Inventor： Ke Chen ,  Nikolai Smolyanskiy ,  Alexey Kamenev ,  Ryan Oldja ,  Tilman Wekel ,  David Nister ,  Joachim Pehserl ,  Ibrahim Eden ,  Sangmin Oh ,  Ruchi Bhargava

IPC: G05D1/00 ,  G06T7/10 ,  G06T5/50 ,  G06K9/00 ,  G06K9/62

Abstract: A deep neural network(s) (DNN) may be used to perform panoptic segmentation by performing pixel-level class and instance segmentation of a scene using a single pass of the DNN. Generally, one or more images and/or other sensor data may be stitched together, stacked, and/or combined, and fed into a DNN that includes a common trunk and several heads that predict different outputs. The DNN may include a class confidence head that predicts a confidence map representing pixels that belong to particular classes, an instance regression head that predicts object instance data for detected objects, an instance clustering head that predicts a confidence map of pixels that belong to particular instances, and/or a depth head that predicts range values. These outputs may be decoded to identify bounding shapes, class labels, instance labels, and/or range values for detected objects, and used to enable safe path planning and control of an autonomous vehicle.

公开(公告)号：US20190295282A1

公开(公告)日：2019-09-26

申请号：US16356439

申请日：2019-03-18

Applicant: NVIDIA Corporation

Inventor： Nikolai Smolyanskiy ,  Alexey Kamenev ,  Stan Birchfield

IPC: G06T7/593 ,  G06N3/08 ,  G06N3/04 ,  G06K9/62 ,  G06T1/20

Abstract: Various examples of the present disclosure include a stereoscopic deep neural network (DNN) that produces accurate and reliable results in real-time. Both LIDAR data (supervised training) and photometric error (unsupervised training) may be used to train the DNN in a semi-supervised manner. The stereoscopic DNN may use an exponential linear unit (ELU) activation function to increase processing speeds, as well as a machine learned argmax function that may include a plurality of convolutional layers having trainable parameters to account for context. The stereoscopic DNN may further include layers having an encoder/decoder architecture, where the encoder portion of the layers may include a combination of three-dimensional convolutional layers followed by two-dimensional convolutional layers.

公开(公告)号：US20230013338A1

公开(公告)日：2023-01-19

申请号：US17855233

申请日：2022-06-30

Applicant: NVIDIA Corporation

Inventor： Nikolai Smolyanskiy ,  Alexey Kamenev ,  Jeffrey David Smith ,  Stanley Thomas Birchfield

IPC: G05D1/00 ,  G05D1/02 ,  G06K9/00 ,  G06K9/62 ,  G06V20/56 ,  B62D15/02 ,  G06V10/94 ,  G06N3/04 ,  G06N7/00 ,  G05D1/10 ,  G06N3/08 ,  B62D6/00

Abstract: A method, computer readable medium, and system are disclosed for performing autonomous path navigation using deep neural networks. The method includes the steps of receiving image data at a deep neural network (DNN), determining, by the DNN, both an orientation of a vehicle with respect to a path and a lateral position of the vehicle with respect to the path, utilizing the image data, and controlling a location of the vehicle, utilizing the orientation of the vehicle with respect to the path and the lateral position of the vehicle with respect to the path.

公开(公告)号：US20210295171A1

公开(公告)日：2021-09-23

申请号：US16824199

申请日：2020-03-19

Applicant: NVIDIA Corporation

Inventor： Alexey Kamenev ,  Nikolai Smolyanskiy ,  Ishwar Kulkarni ,  Ollin Boer Bohan ,  Fangkai Yang ,  Alperen Degirmenci ,  Ruchi Bhargava ,  Urs Muller ,  David Nister ,  Rotem Aviv

IPC: G06N3/08 ,  G06N3/04

Abstract: In various examples, past location information corresponding to actors in an environment and map information may be applied to a deep neural network (DNN)—such as a recurrent neural network (RNN)—trained to compute information corresponding to future trajectories of the actors. The output of the DNN may include, for each future time slice the DNN is trained to predict, a confidence map representing a confidence for each pixel that an actor is present and a vector field representing locations of actors in confidence maps for prior time slices. The vector fields may thus be used to track an object through confidence maps for each future time slice to generate a predicted future trajectory for each actor. The predicted future trajectories, in addition to tracked past trajectories, may be used to generate full trajectories for the actors that may aid an ego-vehicle in navigating the environment.

公开(公告)号：US10705525B2

公开(公告)日：2020-07-07

申请号：US15939116

申请日：2018-03-28

Applicant: NVIDIA Corporation

Inventor： Nikolai Smolyanskiy ,  Alexey Kamenev ,  Jeffrey David Smith ,  Stanley Thomas Birchfield

IPC: G05D1/00 ,  G05D1/02 ,  B62D6/00 ,  G06N3/08 ,  G06N7/00 ,  B62D15/02 ,  G06K9/00 ,  G06K9/62 ,  G05D1/10 ,  G06N3/04

Abstract: A method, computer readable medium, and system are disclosed for performing autonomous path navigation using deep neural networks. The method includes the steps of receiving image data at a deep neural network (DNN), determining, by the DNN, both an orientation of a vehicle with respect to a path and a lateral position of the vehicle with respect to the path, utilizing the image data, and controlling a location of the vehicle, utilizing the orientation of the vehicle with respect to the path and the lateral position of the vehicle with respect to the path.

公开(公告)号：US20180292825A1

公开(公告)日：2018-10-11

申请号：US15939116

申请日：2018-03-28

Applicant: NVIDIA Corporation

Inventor： Nikolai Smolyanskiy ,  Alexey Kamenev ,  Jeffrey David Smith ,  Stanley Thomas Birchfield

IPC: G05D1/00 ,  G05D1/02 ,  B62D6/00 ,  G06N3/08 ,  G06N7/00

CPC classification number: G05D1/0088 ,  B62D6/001 ,  B62D15/025 ,  G05D1/0221 ,  G05D1/024 ,  G05D1/0242 ,  G05D1/0246 ,  G05D1/0255 ,  G05D1/0257 ,  G05D1/0268 ,  G05D1/102 ,  G05D2201/0213 ,  G06K9/00 ,  G06K9/00791 ,  G06K9/00986 ,  G06K9/6273 ,  G06N3/04 ,  G06N3/08 ,  G06N3/084 ,  G06N7/005

Abstract: A method, computer readable medium, and system are disclosed for performing autonomous path navigation using deep neural networks. The method includes the steps of receiving image data at a deep neural network (DNN), determining, by the DNN, both an orientation of a vehicle with respect to a path and a lateral position of the vehicle with respect to the path, utilizing the image data, and controlling a location of the vehicle, utilizing the orientation of the vehicle with respect to the path and the lateral position of the vehicle with respect to the path.