公开(公告)号：US20250118010A1

公开(公告)日：2025-04-10

申请号：US18903411

申请日：2024-10-01

Applicant: NEC Laboratories America, Inc.

Inventor： Ziyu Jiang ,  Bingbing Zhuang ,  Manmohan Chandraker

IPC: G06T15/08 ,  G06T15/06 ,  G06T15/50

Abstract: A computer-implemented method for synthesizing an image includes capturing data from a scene and decomposing the captured scene into static objects; dynamic objects and sky. Bounding boxes are generated for the dynamic objects and motion is simulated for the dynamic objects as static movement of the bounding boxes. The dynamic objects and the static objects are merged according to density and color of sample points. The sky is blended into a merged version of the dynamic objects and the static objects, and an image is synthesized from volume rendered rays.

公开(公告)号：US20250148736A1

公开(公告)日：2025-05-08

申请号：US18924258

申请日：2024-10-23

Applicant: NEC Laboratories America, Inc.

Inventor： Bingbing Zhuang ,  Ziyu Jiang ,  Manmohan Chandraker ,  Shanlin Sun

IPC: G06T19/20 ,  G06T1/20 ,  G06T7/246 ,  G06T7/90 ,  G06T15/00 ,  G06T15/06 ,  G06T15/20 ,  G06T15/50

Abstract: A computer-implemented method for synthesizing an image includes extracting agent neural radiance fields (NeRFs) from driving video logs and storing agent NeRFs in a database. For a driving video log to be edited, a scene NeRF and agent NeRFs are extracted from the driving video log to be edited. One or more agent NeRFs are selected from the database to insert into or replace existing agents in a traffic scene of the driving video log based on photorealism criteria. The traffic scene is edited by inserting a selected agent NeRF into the traffic scene, replacing existing agents in the traffic scene with the selected agent NeRF, or removing one or more existing agents from the traffic scene. An image of the edited traffic scene is synthesized by composing edited agent NeRFs with the scene NeRF and performing volume rendering.

公开(公告)号：US20250148697A1

公开(公告)日：2025-05-08

申请号：US18936290

申请日：2024-11-04

Applicant: NEC Laboratories America, Inc.

Inventor： Ziyu Jiang ,  Bingbing Zhuang ,  Manmohan Chandraker

IPC: G06T15/08

Abstract: Methods and systems include training a model for rendering a three-dimensional volume using a loss function that includes a depth loss term and a distribution loss term that regularize an output of the model to produce realistic scenarios. A simulated scenario is generated based on an original scenario, with the simulated scenario including a different position and pose relative to the original scenario in a three-dimensional (3D) scene that is generated by the model from the original scenario. A self-driving model is trained for an autonomous vehicle using the simulated scenario.

公开(公告)号：US20250118009A1

公开(公告)日：2025-04-10

申请号：US18903348

申请日：2024-10-01

Applicant: NEC Laboratories America, Inc.

Inventor： Bingbing Zhuang ,  Ziyu Jiang ,  Buyu Liu ,  Manmohan Chandraker ,  Shanlin Sun

IPC: G06T15/08 ,  G01S17/89 ,  G06V10/774 ,  G06V20/52

Abstract: A computer-implemented method for synthesizing an image includes capturing data from a scene and fusing grid-based representations of the scene from different encodings to inherit beneficial properties of the different encodings, The encodings include Lidar encoding and a high definition map encoding. Rays are rendered from fused grid-based representations. A density and color are determined for points in the rays. A volume rendering is employed for the rays with the density and color. An image is synthesized from the volume rendered rays with the density and the color.