DYNAMIC LAYER PARTITIONING FOR INCREMENTAL TRAINING OF NEURAL RADIANCE FIELDS

    公开(公告)号:US20240355111A1

    公开(公告)日:2024-10-24

    申请号:US18758110

    申请日:2024-06-28

    CPC classification number: G06V10/82 G06T7/20 G06T7/90 G06T2207/10024

    Abstract: Example apparatus disclosed herein are to train a neural network based on initial video frames of an input video to generate neural representations of the initial video frames, the neural network having a first group of layers and a second group of layers, the first group of layers to be retrained for subsequent video frames after the initial video frames, the second group of layers to be selectively frozen for the subsequent video frames. Disclosed example apparatus are also to select a layer of the second group of layers to be unfrozen for a first video frame subsequent to the initial video frames, and retrain the first group of layers and the selected layer of the second group of layers to generate a neural representation of the first video frame, unselected ones of the second group of layers to remain frozen in the neural representation of the first video frame.

    View interpolation for visual storytelling

    公开(公告)号:US10701282B2

    公开(公告)日:2020-06-30

    申请号:US14749483

    申请日:2015-06-24

    Abstract: A plurality of frames of a video recorded by a video camera and depth maps of the plurality of frames are stored in a data storage. One or more target video camera positions are determined. Each frame of the plurality of frames is associated with one or more of the target video camera positions. For each frame, one or more synthesized frames from the viewpoint of the one or more target camera positions associated with that frame are generated by applying a view interpolation algorithm to that frame using the color pixels of that frame and the depth map of that frame. Users can provide their input about the new camera positions and other camera parameters through multiple input modalities. The synthesized frames are concatenated to create a modified video. Other embodiments are also described and claimed.

    HEAD-MOUNTED DISPLAY DEVICE
    4.
    发明申请

    公开(公告)号:US20180199028A1

    公开(公告)日:2018-07-12

    申请号:US15402866

    申请日:2017-01-10

    CPC classification number: H04N13/344 G02B27/0172 G02B2027/0134

    Abstract: In one example, a head-mounted display (HMD) device includes multiple display panels arranged in parallel with each other. Each of the display panels is associated with one of multiple focal lengths. The HMD device includes multiple lenses to view a three-dimensional (3D) scene on the display panels. The HMD device also includes a controller to provide a frame of the 3D scene, viewable at the focal lengths. The frames include focal layers generated at one of the focal lengths. The frames are rendered by displaying the focal layers in a sequence on the display panels associated with the focal length at which the focal layer is generated. The controller also allows visible light to pass through one or more of the display panels based on whether the render planes are between an active focal layer and the lenses.

    Filling disparity holes based on resolution decoupling

    公开(公告)号:US09998723B2

    公开(公告)日:2018-06-12

    申请号:US14607355

    申请日:2015-01-28

    CPC classification number: H04N13/128 H04N2013/0081

    Abstract: Techniques related to filling hole regions in a disparity map are discussed. Such techniques may include generating approximated disparity values for grid nodes of an approximation grid having a lower resolution than the disparity map based on disparity values within outer hole contour regions circumscribing the hole regions. The hole regions may be filled with upsampled disparity values based on the approximated disparity values to provide a final disparity map.

    THREE DIMENSIONAL GAUSSIAN SPLATTING INITIALIZATION BASED ON TRAINED NEURAL RADIANCE FIELD REPRESENTATIONS

    公开(公告)号:US20240355047A1

    公开(公告)日:2024-10-24

    申请号:US18756600

    申请日:2024-06-27

    CPC classification number: G06T17/00 G06T7/90 G06T2207/10024

    Abstract: Example systems, apparatus, articles of manufacture, and methods are disclosed to implement three dimensional gaussian splatting initialization based on trained neural radiance field representations. Example apparatus disclosed herein determine a location for an initial three-dimensional (3D) gaussian splat based on optical densities obtained from a trained neural representation of a scene, the optical densities associated with location sample points along a training ray used to train the neural representation. Disclosed example apparatus also set parameters of the initial 3D gaussian splat based on one of the optical densities associated with the location of the initial 3D gaussian splat and a color value obtained from the trained neural representation, the color value associated with the location of the initial 3D gaussian splat, the initial 3D gaussian splat to be used to generate a 3D gaussian splat representation of the scene.

    VIEW INTERPOLATION FOR VISUAL STORYTELLING
    9.
    发明申请
    VIEW INTERPOLATION FOR VISUAL STORYTELLING 审中-公开
    查看视觉识别插值

    公开(公告)号:US20160381341A1

    公开(公告)日:2016-12-29

    申请号:US14749483

    申请日:2015-06-24

    Abstract: A plurality of frames of a video recorded by a video camera and depth maps of the plurality of frames are stored in a data storage. One or more target video camera positions are determined. Each frame of the plurality of frames is associated with one or more of the target video camera positions. For each frame, one or more synthesized frames from the viewpoint of the one or more target camera positions associated with that frame are generated by applying a view interpolation algorithm to that frame using the color pixels of that frame and the depth map of that frame. Users can provide their input about the new camera positions and other camera parameters through multiple input modalities. The synthesized frames are concatenated to create a modified video. Other embodiments are also described and claimed.

    Abstract translation: 由摄像机记录的视频的多个帧和多个帧的深度图被存储在数据存储器中。 确定一个或多个目标摄像机位置。 多个帧中的每个帧与目标摄像机位置中的一个或多个相关联。 对于每个帧,通过使用该帧的颜色像素和该帧的深度图对该帧应用视图插值算法来生成与该帧相关联的一个或多个目标摄像机位置的视点的一个或多个合成帧。 用户可以通过多种输入模式提供有关新相机位置和其他相机参数的输入。 合成的帧被级联以创建修改的视频。 还描述和要求保护其他实施例。

Patent Agency Ranking