-
公开(公告)号:US20250118009A1
公开(公告)日:2025-04-10
申请号:US18903348
申请日:2024-10-01
Applicant: NEC Laboratories America, Inc.
Inventor: Bingbing Zhuang , Ziyu Jiang , Buyu Liu , Manmohan Chandraker , Shanlin Sun
IPC: G06T15/08 , G01S17/89 , G06V10/774 , G06V20/52
Abstract: A computer-implemented method for synthesizing an image includes capturing data from a scene and fusing grid-based representations of the scene from different encodings to inherit beneficial properties of the different encodings, The encodings include Lidar encoding and a high definition map encoding. Rays are rendered from fused grid-based representations. A density and color are determined for points in the rays. A volume rendering is employed for the rays with the density and color. An image is synthesized from the volume rendered rays with the density and the color.
-
公开(公告)号:US20250115278A1
公开(公告)日:2025-04-10
申请号:US18905695
申请日:2024-10-03
Applicant: NEC Laboratories America, Inc.
Inventor: Francesco Pittaluga , Buyu Liu , Manmohan Chandraker , Kaiyuan Zhang
IPC: B60W60/00
Abstract: Systems and methods for generating adversarial driving scenarios for autonomous vehicles. An artificial intelligence model can compute an adversarial loss function by minimizing the distance between predicted adversarial perturbed trajectories and corresponding generated neighbor future trajectories from input data. A traffic violation loss function can be computed based on observed adversarial agents adhering to driving rules from the input data. A comfort loss function can be computed based on the predicted driving characteristics of adversarial vehicles relevant to comfort of hypothetical passengers from the input data. A planner module can be trained for autonomous vehicles based on a combined loss function of the adversarial loss function, the traffic violation loss function and the comfort loss function to generate adversarial driving scenarios. An autonomous vehicle can be controlled based on trajectories generated in the adversarial driving scenarios.
-
公开(公告)号:US20240351582A1
公开(公告)日:2024-10-24
申请号:US18639307
申请日:2024-04-18
Applicant: NEC Laboratories America, Inc.
Inventor: Buyu Liu , Sriram Nochur Narayanan , Bingbing Zhuang , Yumin Suh
IPC: B60W30/09 , B60W30/095 , B60W50/00 , G06V20/56
CPC classification number: B60W30/09 , B60W30/0956 , B60W50/0097 , G06V20/588
Abstract: Methods and systems for trajectory prediction include encoding trajectories of agents in a scene from past images of the scene. Lane centerlines are encoded for agents in the scene. The agents in the scene are encoded using the encoded trajectories and the encoded lane centerlines. A hypercolumn trajectory is decoded from the encoded agents to generate predicted trajectories for the agents. A vehicle is automatically operated responsive to the predicted trajectories.
-
公开(公告)号:US20220144256A1
公开(公告)日:2022-05-12
申请号:US17521139
申请日:2021-11-08
Applicant: NEC Laboratories America, Inc.
Inventor: Sriram Nochur Narayanan , Ramin Moslemi , Francesco Pittaluga , Buyu Liu , Manmohan Chandraker
IPC: B60W30/09 , G06F16/29 , G06N3/08 , B60W30/095 , G08G1/16
Abstract: A method for driving path prediction is provided. The method concatenates past trajectory features and lane centerline features in a channel dimension at an agent's respective location in a top view map to obtain concatenated features thereat. The method obtains convolutional features derived from the top view map, the concatenated features, and a single representation of the training scene the vehicle and agent interactions. The method extracts hypercolumn descriptor vectors which include the convolutional features from the agent's respective location in the top view map. The method obtains primary and auxiliary trajectory predictions from the hypercolumn descriptor vectors. The method generates a respective score for each of the primary and auxiliary trajectory predictions. The method trains a vehicle trajectory prediction neural network using a reconstruction loss, a regularization loss objective, and an IOC loss objective responsive to the respective score for each of the primary and auxiliary trajectory predictions.
-
公开(公告)号:US20210110210A1
公开(公告)日:2021-04-15
申请号:US17128535
申请日:2020-12-21
Applicant: NEC Laboratories America, Inc.
Inventor: Yi-Hsuan Tsai , Kihyuk Sohn , Buyu Liu , Manmohan Chandraker , Jong-Chyi Su
IPC: G06K9/62 , G06K9/00 , G06K9/32 , B60W30/095 , B60W30/09 , B60W10/20 , B60W10/18 , B60W50/00 , G08G1/16 , G06N3/08
Abstract: Systems and methods for lane marking and road sign recognition are provided. The system aligns image level features between a source domain and a target domain based on an adversarial learning process while training a domain discriminator. The target domain includes one or more road scenes having lane markings and road signs. The system selects, using the domain discriminator, unlabeled samples from the target domain that are far away from existing annotated samples from the target domain. The system selects, based on a prediction score of each of the unlabeled samples, samples with lower prediction scores. The system annotates the samples with the lower prediction scores.
-
公开(公告)号:US12254681B2
公开(公告)日:2025-03-18
申请号:US17903393
申请日:2022-09-06
Applicant: NEC Laboratories America, Inc.
Inventor: Yi-Hsuan Tsai , Bingbing Zhuang , Samuel Schulter , Buyu Liu , Sparsh Garg , Ramin Moslemi , Inkyu Shin
IPC: G06K9/00 , G01S17/89 , G06V10/776 , G06V10/80
Abstract: Systems and methods are provided for multi-modal test-time adaptation. The method includes inputting a digital image into a pre-trained Camera Intra-modal Pseudo-label Generator, and inputting a point cloud set into a pre-trained Lidar Intra-modal Pseudo-label Generator. The method further includes applying a fast 2-dimension (2D) model, and a slow 2D model, to the inputted digital image to apply pseudo-labels, and applying a fast 3-dimension (3D) model, and a slow 3D model, to the inputted point cloud set to apply pseudo-labels. The method further includes fusing pseudo-label predictions from the fast models and the slow models through an Inter-modal Pseudo-label Refinement module to obtain robust pseudo labels, and measuring a prediction consistency for the pseudo-labels. The method further includes selecting confident pseudo-labels from the robust pseudo labels and measured prediction consistencies to form a final cross-modal pseudo-label set as a self-training signal, and updating batch parameters utilizing the self-training signal.
-
公开(公告)号:US11610420B2
公开(公告)日:2023-03-21
申请号:US17128565
申请日:2020-12-21
Applicant: NEC Laboratories America, Inc.
Inventor: Yi-Hsuan Tsai , Kihyuk Sohn , Buyu Liu , Manmohan Chandraker , Jong-Chyi Su
Abstract: Systems and methods for human detection are provided. The system aligns image level features between a source domain and a target domain based on an adversarial learning process while training a domain discriminator. The target domain includes humans in one or more different scenes. The system selects, using the domain discriminator, unlabeled samples from the target domain that are far away from existing annotated samples from the target domain. The system selects, based on a prediction score of each of the unlabeled samples, samples with lower prediction scores. The system annotates the samples with the lower prediction scores.
-
公开(公告)号:US11373067B2
公开(公告)日:2022-06-28
申请号:US16526073
申请日:2019-07-30
Applicant: NEC Laboratories America, Inc.
Inventor: Samuel Schulter , Ziyan Wang , Buyu Liu , Manmohan Chandraker
IPC: G06K9/62 , B60R11/04 , G05D1/02 , G06N3/02 , G06V20/56 , H04N5/32 , B60W50/14 , B60W60/00 , G06N3/04 , G06N3/08 , G06V10/82 , H04N5/232
Abstract: A method for implementing parametric models for scene representation to improve autonomous task performance includes generating an initial map of a scene based on at least one image corresponding to a perspective view of the scene, the initial map including a non-parametric top-view representation of the scene, implementing a parametric model to obtain a scene element representation based on the initial map, the scene element representation providing a description of one or more scene elements of the scene and corresponding to an estimated semantic layout of the scene, identifying one or more predicted locations of the one or more scene elements by performing three-dimensional localization based on the at least one image, and obtaining an overlay for performing an autonomous task by placing the one or more scene elements with the one or more respective predicted locations onto the scene element representation.
-
公开(公告)号:US20220111869A1
公开(公告)日:2022-04-14
申请号:US17494927
申请日:2021-10-06
Applicant: NEC Laboratories America, Inc.
Inventor: Buyu Liu , Pan Ji , Bingbing Zhuang , Manmohan Chandraker , Uday Kusupati
IPC: B60W60/00 , G06T7/50 , G06K9/72 , G06T7/10 , G06K9/00 , G06K9/62 , G06T7/70 , G06N3/04 , G06N3/08
Abstract: Methods and systems for determining a path include detecting objects within a perspective image that shows a scene. Depth is predicted within the perspective image. Semantic segmentation is performed on the perspective image. An attention map is generated using the detected objects and the predicted depth. A refined top-down view of the scene is generated using the predicted depth and the semantic segmentation. A parametric top-down representation of the scene is determined using a relational graph model. A path through the scene is determined using the parametric top-down representation.
-
公开(公告)号:US20210110209A1
公开(公告)日:2021-04-15
申请号:US17128612
申请日:2020-12-21
Applicant: NEC Laboratories America, Inc.
Inventor: Yi-Hsuan Tsai , Kihyuk Sohn , Buyu Liu , Manmohan Chandraker , Jong-Chyi Su
Abstract: Systems and methods for construction zone segmentation are provided. The system aligns image level features between a source domain and a target domain based on an adversarial learning process while training a domain discriminator. The target domain includes construction zones scenes having various objects. The system selects, using the domain discriminator, unlabeled samples from the target domain that are far away from existing annotated samples from the target domain. The system selects, based on a prediction score of each of the unlabeled samples, samples with lower prediction scores. The system annotates the samples with the lower prediction scores.
-
-
-
-
-
-
-
-
-