EDITABLE FREE-VIEWPOINT VIDEO USING A LAYERED NEURAL REPRESENTATION

    公开(公告)号:US20240290059A1

    公开(公告)日:2024-08-29

    申请号:US18571748

    申请日:2021-07-26

    IPC分类号: G06V10/25 G06T3/02 G06T7/50

    CPC分类号: G06V10/25 G06T3/02 G06T7/50

    摘要: A computer-implemented method of generating editable free-viewport videos is provided. A plurality of video of a scene from a plurality of views is obtained. The scene comprises includes an environment and one or more dynamic entities. A 3D bounding-box is generated for each dynamic entity in the scene. A computer device encodes a machine learning model including an environment layer and a dynamic entity layer for each dynamic entity in the scene. The environment layer represents a continuous function of space and time of the environment. The dynamic entity layer represents a continuous function of space and time of the dynamic entity. The dynamic entity layer includes a deformation module and a neural radiance module. The deformation module is configured to deform a spatial coordinate in accordance with a timestamp and a trained deformation weight. The neural radiance module is configured to derive a density value and a color.