公开(公告)号：US20250118009A1

公开(公告)日：2025-04-10

申请号：US18903348

申请日：2024-10-01

Applicant: NEC Laboratories America, Inc.

Inventor： Bingbing Zhuang ,  Ziyu Jiang ,  Buyu Liu ,  Manmohan Chandraker ,  Shanlin Sun

IPC: G06T15/08 ,  G01S17/89 ,  G06V10/774 ,  G06V20/52

Abstract: A computer-implemented method for synthesizing an image includes capturing data from a scene and fusing grid-based representations of the scene from different encodings to inherit beneficial properties of the different encodings, The encodings include Lidar encoding and a high definition map encoding. Rays are rendered from fused grid-based representations. A density and color are determined for points in the rays. A volume rendering is employed for the rays with the density and color. An image is synthesized from the volume rendered rays with the density and the color.

公开(公告)号：US20250117029A1

公开(公告)日：2025-04-10

申请号：US18905280

申请日：2024-10-03

Applicant: NEC Laboratories America, Inc.

Inventor： Tom Bu ,  Bingbing Zhuang ,  Manmohan Chandraker

IPC: G05D1/86 ,  G05D1/242 ,  G05D1/243 ,  G05D107/60 ,  G05D111/67

Abstract: Systems and methods for automatic multi-modality sensor calibration with near-infrared images (NIR). Image keypoints from collected images and NIR keypoints from NIR can be detected. A deep-learning-based neural network that learns relation graphs between the image keypoints and the NIR keypoints can match the image keypoints and the NIR keypoints. Three dimensional (3D) points from 3D point cloud data can be filtered based on corresponding 3D points from the NIR keypoints (NIR-to-3D points) to obtain filtered NIR-to-3D points. An extrinsic calibration can be optimized based on a reprojection error computed from the filtered NIR-to-3D points to obtain an optimized extrinsic calibration for an autonomous entity control system. An entity can be controlled by employing the optimized extrinsic calibration for the autonomous entity control system.

公开(公告)号：US20240351582A1

公开(公告)日：2024-10-24

申请号：US18639307

申请日：2024-04-18

Applicant: NEC Laboratories America, Inc.

Inventor： Buyu Liu ,  Sriram Nochur Narayanan ,  Bingbing Zhuang ,  Yumin Suh

IPC: B60W30/09 ,  B60W30/095 ,  B60W50/00 ,  G06V20/56

CPC classification number: B60W30/09 ,  B60W30/0956 ,  B60W50/0097 ,  G06V20/588

Abstract: Methods and systems for trajectory prediction include encoding trajectories of agents in a scene from past images of the scene. Lane centerlines are encoded for agents in the scene. The agents in the scene are encoded using the encoded trajectories and the encoded lane centerlines. A hypercolumn trajectory is decoded from the encoded agents to generate predicted trajectories for the agents. A vehicle is automatically operated responsive to the predicted trajectories.

公开(公告)号：US11222409B2

公开(公告)日：2022-01-11

申请号：US16867805

申请日：2020-05-06

Applicant: NEC Laboratories America, Inc.

Inventor： Quoc-Huy Tran ,  Bingbing Zhuang ,  Pan Ji ,  Manmohan Chandraker

IPC: G06T5/00 ,  G06T7/50 ,  G06N3/04

Abstract: A method for correcting blur effects is presented. The method includes generating a plurality of images from a camera, synthesizing blurred images from sharp image counterparts to generate training data to train a structure-and-motion-aware convolutional neural network (CNN), and predicting a camera motion and a depth map from a single blurred image by employing the structure-and-motion-aware CNN to remove blurring from the single blurred image.

公开(公告)号：US20200234467A1

公开(公告)日：2020-07-23

申请号：US16736451

申请日：2020-01-07

Applicant: NEC Laboratories America, Inc.

Inventor： Quoc-Huy Tran ,  Bingbing Zhuang ,  Pan Ji ,  Manmohan Chandraker

IPC: G06T7/80 ,  G06T7/64 ,  G06T5/00 ,  G06N3/08

Abstract: Systems and methods for camera self-calibration are provided. The method includes receiving real uncalibrated images, and estimating, using a camera self-calibration network, multiple predicted camera parameters corresponding to the real uncalibrated images. Deep supervision is implemented based on a dependence order between the plurality of predicted camera parameters to place supervision signals across multiple layers according to the dependence order. The method also includes determining calibrated images using the real uncalibrated images and the predicted camera parameters.

公开(公告)号：US12254681B2

公开(公告)日：2025-03-18

申请号：US17903393

申请日：2022-09-06

Applicant: NEC Laboratories America, Inc.

Inventor： Yi-Hsuan Tsai ,  Bingbing Zhuang ,  Samuel Schulter ,  Buyu Liu ,  Sparsh Garg ,  Ramin Moslemi ,  Inkyu Shin

IPC: G06K9/00 ,  G01S17/89 ,  G06V10/776 ,  G06V10/80

Abstract: Systems and methods are provided for multi-modal test-time adaptation. The method includes inputting a digital image into a pre-trained Camera Intra-modal Pseudo-label Generator, and inputting a point cloud set into a pre-trained Lidar Intra-modal Pseudo-label Generator. The method further includes applying a fast 2-dimension (2D) model, and a slow 2D model, to the inputted digital image to apply pseudo-labels, and applying a fast 3-dimension (3D) model, and a slow 3D model, to the inputted point cloud set to apply pseudo-labels. The method further includes fusing pseudo-label predictions from the fast models and the slow models through an Inter-modal Pseudo-label Refinement module to obtain robust pseudo labels, and measuring a prediction consistency for the pseudo-labels. The method further includes selecting confident pseudo-labels from the robust pseudo labels and measured prediction consistencies to form a final cross-modal pseudo-label set as a self-training signal, and updating batch parameters utilizing the self-training signal.

公开(公告)号：US20240354921A1

公开(公告)日：2024-10-24

申请号：US18616396

申请日：2024-03-26

Applicant: NEC Laboratories America, Inc.

Inventor： Sparsh Garg ,  Bingbing Zhuang ,  Samuel Schulter ,  Manmohan Chandraker

IPC: G06T7/00 ,  G06T7/10 ,  G06T7/50 ,  G06V20/56

CPC classification number: G06T7/0002 ,  G06T7/10 ,  G06T7/50 ,  G06V20/588 ,  G06T2207/10028 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/30256

Abstract: Systems and methods for road defect level prediction. A depth map is obtained from an image dataset received from input peripherals by employing a vision transformer model. A plurality of semantic maps is obtained from the image dataset by employing a semantic segmentation model to give pixel-wise segmentation results of road scenes to detect road pixels. Regions of interest (ROI) are detected by utilizing the road pixels. Road defect levels are predicted by fitting the ROI and the depth map into a road surface model to generate road points classified into road defect levels. The predicted road defect levels are visualized on a road map.

公开(公告)号：US20240303365A1

公开(公告)日：2024-09-12

申请号：US18598198

申请日：2024-03-07

Applicant: NEC Laboratories America, Inc.

Inventor： Francesco Pittaluga ,  Bingbing Zhuang ,  Xiang Yu

IPC: G06F21/62 ,  G06V10/75

CPC classification number: G06F21/6227 ,  G06V10/751

Abstract: Systems and methods are provided for privacy-preserving image feature matching in computer vision applications, including receiving a raw image descriptor, and perturbing the raw image descriptor using a subset selection mechanism to generate a perturbed descriptor set that includes the raw image descriptor and additional descriptors. Each descriptor in the perturbed descriptor set is replaced with its nearest neighbor in a predefined descriptor database to reduce the output domain size of the subset selection mechanism. Local differential privacy (LDP) protocols are employed to further perturb the descriptor set, ensuring formal privacy guarantees, and the perturbed descriptor set is matched against a second set of descriptors for image feature matching.

公开(公告)号：US11694311B2

公开(公告)日：2023-07-04

申请号：US17182836

申请日：2021-02-23

Applicant: NEC Laboratories America, Inc.

Inventor： Bingbing Zhuang ,  Quoc-Huy Tran

IPC: G06T5/00 ,  G06T7/20 ,  H04N23/68

CPC classification number: G06T5/003 ,  G06T7/20 ,  H04N23/689

Abstract: A computer-implemented method executed by at least one processor for applying rolling shutter (RS)-aware spatially varying differential homography fields for simultaneous RS distortion removal and image stitching is presented. The method includes inputting two consecutive frames including RS distortions from a video stream, performing keypoint detection and matching to extract correspondences between the two consecutive frames, feeding the correspondences between the two consecutive frames into an RS-aware differential homography estimation component to filter out outlier correspondences, sending inlier correspondences to an RS-aware spatially varying differential homography field estimation component to compute an RS-aware spatially varying differential homography field, and using the RS-aware spatially varying differential homography field in an RS stitching and correction component to produce stitched images with removal of the RS distortions.

公开(公告)号：US20220111869A1

公开(公告)日：2022-04-14

申请号：US17494927

申请日：2021-10-06

Applicant: NEC Laboratories America, Inc.

Inventor： Buyu Liu ,  Pan Ji ,  Bingbing Zhuang ,  Manmohan Chandraker ,  Uday Kusupati

IPC: B60W60/00 ,  G06T7/50 ,  G06K9/72 ,  G06T7/10 ,  G06K9/00 ,  G06K9/62 ,  G06T7/70 ,  G06N3/04 ,  G06N3/08

Abstract: Methods and systems for determining a path include detecting objects within a perspective image that shows a scene. Depth is predicted within the perspective image. Semantic segmentation is performed on the perspective image. An attention map is generated using the detected objects and the predicted depth. A refined top-down view of the scene is generated using the predicted depth and the semantic segmentation. A parametric top-down representation of the scene is determined using a relational graph model. A path through the scene is determined using the parametric top-down representation.