公开(公告)号：US11182916B2

公开(公告)日：2021-11-23

申请号：US16728598

申请日：2019-12-27

Applicant: NVIDIA Corporation

Inventor： Yilin Yang ,  Bala Siva Sashank Jujjavarapu ,  Pekka Janis ,  Zhaoting Ye ,  Sangmin Oh ,  Minwoo Park ,  Daniel Herrera Castro ,  Tommi Koivisto ,  David Nister

IPC: G06K9/00 ,  G06T7/536

Abstract: In various examples, a deep neural network (DNN) is trained to accurately predict, in deployment, distances to objects and obstacles using image data alone. The DNN may be trained with ground truth data that is generated and encoded using sensor data from any number of depth predicting sensors, such as, without limitation, RADAR sensors, LIDAR sensors, and/or SONAR sensors. Camera adaptation algorithms may be used in various embodiments to adapt the DNN for use with image data generated by cameras with varying parameters—such as varying fields of view. In some examples, a post-processing safety bounds operation may be executed on the predictions of the DNN to ensure that the predictions fall within a safety-permissible range.

公开(公告)号：US20190258878A1

公开(公告)日：2019-08-22

申请号：US16277895

申请日：2019-02-15

Applicant: NVIDIA Corporation

Inventor： Tommi Koivisto ,  Pekka Janis ,  Tero Kuosmanen ,  Timo Roman ,  Sriya Sarathy ,  William Zhang ,  Nizar Assaf ,  Colin Tracey

IPC: G06K9/00 ,  G06K9/46 ,  G06K9/62

Abstract: In various examples, detected object data representative of locations of detected objects in a field of view may be determined. One or more clusters of the detected objects may be generated based at least in part on the locations and features of the cluster may be determined for use as inputs to a machine learning model(s). A confidence score, computed by the machine learning model(s) based at least in part on the inputs, may be received, where the confidence score may be representative of a probability that the cluster corresponds to an object depicted at least partially in the field of view. Further examples provide approaches for determining ground truth data for training object detectors, such as for determining coverage values for ground truth objects using associated shapes, and for determining soft coverage values for ground truth objects.

公开(公告)号：US20160218782A1

公开(公告)日：2016-07-28

申请号：US15005602

申请日：2016-01-25

Applicant: Nvidia Corporation

Inventor： Pekka Janis ,  Tommi Koivisto ,  Kari Hamalainen

IPC: H04B7/04 ,  H04W16/28

CPC classification number: H04B7/0469 ,  H04B7/0478 ,  H04B7/0639 ,  H04B7/065 ,  H04W16/28

Abstract: A communications system has a cellular structure and the communications system includes a base station that is located within a cell of the cellular structure and employs a Kronecker product of azimuth and elevation precoding vectors for beamforming. Additionally, the communications system includes user equipment that is located within the cell and coupled to the base station to receive a reference channel state information process employing a reference precoding vector for use in a non-reference channel state information process to derive a compensated channel quality indication. A method of operating a communications system is also included.

Abstract translation: 通信系统具有蜂窝结构，并且通信系统包括位于蜂窝结构的小区内的基站，并且使用用于波束成形的方位角和仰角预编码向量的克罗内克积。 此外，通信系统包括位于小区内并耦合到基站的用户设备，以接收使用参考预编码向量的参考信道状态信息过程，以用于非参考信道状态信息处理以导出补偿信道质量 指示。 还包括操作通信系统的方法。

公开(公告)号：US12093824B2

公开(公告)日：2024-09-17

申请号：US18343291

申请日：2023-06-28

Applicant: NVIDIA Corporation

Inventor： Yilin Yang ,  Bala Siva Sashank Jujjavarapu ,  Pekka Janis ,  Zhaoting Ye ,  Sangmin Oh ,  Minwoo Park ,  Daniel Herrera Castro ,  Tommi Koivisto ,  David Nister

IPC: G06K9/00 ,  B60W30/14 ,  B60W60/00 ,  G06F18/214 ,  G06N3/08 ,  G06V10/762 ,  G06V20/56

CPC classification number: G06N3/08 ,  B60W30/14 ,  B60W60/0011 ,  G06F18/2155 ,  G06V10/763 ,  G06V20/56

Abstract: In various examples, a deep neural network (DNN) is trained to accurately predict, in deployment, distances to objects and obstacles using image data alone. The DNN may be trained with ground truth data that is generated and encoded using sensor data from any number of depth predicting sensors, such as, without limitation, RADAR sensors, LIDAR sensors, and/or SONAR sensors. Camera adaptation algorithms may be used in various embodiments to adapt the DNN for use with image data generated by cameras with varying parameters—such as varying fields of view. In some examples, a post-processing safety bounds operation may be executed on the predictions of the DNN to ensure that the predictions fall within a safety-permissible range.

公开(公告)号：US20240192320A1

公开(公告)日：2024-06-13

申请号：US18582358

申请日：2024-02-20

Applicant: NVIDIA Corporation

Inventor： Tommi Koivisto ,  Pekka Janis ,  Tero Kuosmanen ,  Timo Roman ,  Sriya Sarathy ,  William Zhang ,  Nizar Assaf ,  Colin Tracey

IPC: G01S7/41 ,  B60W50/00 ,  G01S7/48 ,  G01S13/86 ,  G01S13/931 ,  G01S17/931 ,  G06F16/35 ,  G06F18/21 ,  G06F18/214 ,  G06F18/23 ,  G06F18/2413 ,  G06N3/044 ,  G06N3/045 ,  G06N3/047 ,  G06N3/048 ,  G06N3/084 ,  G06N20/00 ,  G06V10/20 ,  G06V10/44 ,  G06V10/46 ,  G06V10/762 ,  G06V10/764 ,  G06V10/77 ,  G06V10/774 ,  G06V20/58

CPC classification number: G01S7/417 ,  B60W50/00 ,  G06F16/35 ,  G06F18/214 ,  G06F18/217 ,  G06F18/23 ,  G06F18/2414 ,  G06N3/044 ,  G06N3/045 ,  G06N3/084 ,  G06N20/00 ,  G06V10/255 ,  G06V10/454 ,  G06V10/46 ,  G06V10/762 ,  G06V10/764 ,  G06V10/7715 ,  G06V10/774 ,  G06V20/58 ,  G06V20/584 ,  G01S7/412 ,  G01S7/4802 ,  G01S13/867 ,  G01S2013/9318 ,  G01S2013/9323 ,  G01S17/931 ,  G06N3/047 ,  G06N3/048

Abstract: In various examples, detected object data representative of locations of detected objects in a field of view may be determined. One or more clusters of the detected objects may be generated based at least in part on the locations and features of the cluster may be determined for use as inputs to a machine learning model(s). A confidence score, computed by the machine learning model(s) based at least in part on the inputs, may be received, where the confidence score may be representative of a probability that the cluster corresponds to an object depicted at least partially in the field of view. Further examples provide approaches for determining ground truth data for training object detectors, such as for determining coverage values for ground truth objects using associated shapes, and for determining soft coverage values for ground truth objects.

公开(公告)号：US11579629B2

公开(公告)日：2023-02-14

申请号：US16514404

申请日：2019-07-17

Applicant: NVIDIA Corporation

Inventor： Yue Wu ,  Pekka Janis ,  Xin Tong ,  Cheng-Chieh Yang ,  Minwoo Park ,  David Nister

IPC: G05D1/02 ,  G06K9/62 ,  G06T7/20 ,  G05D1/00 ,  G06V20/10 ,  G06V20/58 ,  G06N3/04

Abstract: In various examples, a sequential deep neural network (DNN) may be trained using ground truth data generated by correlating (e.g., by cross-sensor fusion) sensor data with image data representative of a sequences of images. In deployment, the sequential DNN may leverage the sensor correlation to compute various predictions using image data alone. The predictions may include velocities, in world space, of objects in fields of view of an ego-vehicle, current and future locations of the objects in image space, and/or a time-to-collision (TTC) between the objects and the ego-vehicle. These predictions may be used as part of a perception system for understanding and reacting to a current physical environment of the ego-vehicle.

公开(公告)号：US20220253706A1

公开(公告)日：2022-08-11

申请号：US17723195

申请日：2022-04-18

Applicant: NVIDIA Corporation

Inventor： Yilin Yang ,  Bala Siva Sashank Jujjavarapu ,  Pekka Janis ,  Zhaoting Ye ,  Sangmin Oh ,  Minwoo Park ,  Daniel Herrera Castro ,  Tommi Koivisto ,  David Nister

IPC: G06N3/08 ,  B60W30/14 ,  B60W60/00 ,  G06K9/62 ,  G06V20/56

Abstract: In various examples, a deep neural network (DNN) is trained to accurately predict, in deployment, distances to objects and obstacles using image data alone. The DNN may be trained with ground truth data that is generated and encoded using sensor data from any number of depth predicting sensors, such as, without limitation, RADAR sensors, LIDAR sensors, and/or SONAR sensors. Camera adaptation algorithms may be used in various embodiments to adapt the DNN for use with image data generated by cameras with varying parameters—such as varying fields of view. In some examples, a post-processing safety bounds operation may be executed on the predictions of the DNN to ensure that the predictions fall within a safety-permissible range.

公开(公告)号：US20220108465A1

公开(公告)日：2022-04-07

申请号：US17522624

申请日：2021-11-09

Applicant: NVIDIA Corporation

Inventor： Yilin Yang ,  Bala Siva Jujjavarapu ,  Pekka Janis ,  Zhaoting Ye ,  Sangmin Oh ,  Minwoo Park ,  Daniel Herrera Castro ,  Tommi Koivisto ,  David Nister

IPC: G06T7/536 ,  G06V20/58

Abstract: In various examples, a deep neural network (DNN) is trained—using image data alone—to accurately predict distances to objects, obstacles, and/or a detected free-space boundary. The DNN may be trained with ground truth data that is generated using sensor data representative of motion of an ego-vehicle and/or sensor data from any number of depth predicting sensors—such as, without limitation, RADAR sensors, LIDAR sensors, and/or SONAR sensors. The DNN may be trained using two or more loss functions each corresponding to a particular portion of the environment that depth is predicted for, such that—in deployment—more accurate depth estimates for objects, obstacles, and/or the detected free-space boundary are computed by the DNN. In some embodiments, a sampling algorithm may be used to sample depth values corresponding to an input resolution of the DNN from a predicted depth map of the DNN at an output resolution of the DNN.

公开(公告)号：US20210272304A1

公开(公告)日：2021-09-02

申请号：US16728598

申请日：2019-12-27

Applicant: NVIDIA Corporation

Inventor： Yilin Yang ,  Bala Siva Sashank Jujjavarapu ,  Pekka Janis ,  Zhaoting Ye ,  Sangmin Oh ,  Minwoo Park ,  Daniel Herrera Castro ,  Tommi Koivisto ,  David Nister

IPC: G06T7/536 ,  G06K9/00

Abstract: In various examples, a deep neural network (DNN) is trained to accurately predict, in deployment, distances to objects and obstacles using image data alone. The DNN may be trained with ground truth data that is generated and encoded using sensor data from any number of depth predicting sensors, such as, without limitation, RADAR sensors, LIDAR sensors, and/or SONAR sensors. Camera adaptation algorithms may be used in various embodiments to adapt the DNN for use with image data generated by cameras with varying parameters—such as varying fields of view. In some examples, a post-processing safety bounds operation may be executed on the predictions of the DNN to ensure that the predictions fall within a safety-permissible range.

公开(公告)号：US20160218783A1

公开(公告)日：2016-07-28

申请号：US15005650

申请日：2016-01-25

Applicant: Nvidia Corporation

Inventor： Pekka Janis ,  Tommi Koivisto ,  Kari Hamalainen

IPC: H04B7/04 ,  H04W16/28

CPC classification number: H04B7/0469 ,  H04B7/0478 ,  H04B7/0639 ,  H04B7/065 ,  H04W16/28

Abstract: A communications system has a cellular structure including a base station that is located within a cell of the cellular structure and provides an elevation beamforming transmission based on a set of elevation precoding matrix indicator offsets in an elevation codebook. The communications system also includes user equipment that is located within the cell and coupled to the base station to receive the set of elevation precoding matrix indicator offsets and a set of reference signals to provide channel quality and inter-cell interference measurements, wherein a selected channel quality indicator is based on an increase in channel quality with respect to inter-cell interference at the user equipment and corresponds to one of the set of elevation precoding matrix indicator offsets. A method of operating a communications system having a cellular structure is also provided.

Abstract translation: 通信系统具有蜂窝结构，其包括位于蜂窝结构的小区内的基站，并且基于高程码本中的一组升高预编码矩阵指示符偏移来提供仰角波束成形传输。 通信系统还包括位于小区内并耦合到基站的用户设备，以接收一组升高预编码矩阵指示符偏移和一组参考信号，以提供信道质量和小区间干扰测量，其中所选择的信道 质量指示符是基于用户设备上的小区间干扰的信道质量的增加，并且对应于该组提升预编码矩阵指示符偏移之一。 还提供了一种操作具有蜂窝结构的通信系统的方法。