Invention Grant
- Patent Title: Re-timing objects in video via layered neural rendering
-
Application No.: US17927101Application Date: 2020-05-22
-
Publication No.: US12243145B2Publication Date: 2025-03-04
- Inventor: Forrester H. Cole , Erika Lu , Tali Dekel , William T. Freeman , David Henry Salesin , Michael Rubinstein
- Applicant: Google LLC
- Applicant Address: US CA Mountain View
- Assignee: Google LLC
- Current Assignee: Google LLC
- Current Assignee Address: US CA Mountain View
- Agency: DORITY & MANNING P.A.
- International Application: PCT/US2020/034296 WO 20200522
- International Announcement: WO2021/236104 WO 20211125
- Main IPC: G06T13/80
- IPC: G06T13/80 ; G06V10/44 ; G06V10/82 ; G06V20/40 ; G11B27/00 ; G11B27/031

Abstract:
A computer-implemented method for decomposing videos into multiple layers (212, 213) that can be re-combined with modified relative timings includes obtaining video data including a plurality of image frames (201) depicting one or more objects. For each of the plurality of frames, the computer-implemented method includes generating one or more object maps descriptive of a respective location of at least one object of the one or more objects within the image frame. For each of the plurality of frames, the computer-implemented method includes inputting the image frame and the one or more object maps into a machine-learned layer Tenderer model. (220) For each of the plurality of frames, the computer-implemented method includes receiving, as output from the machine-learned layer Tenderer model, a background layer illustrative of a background of the video data and one or more object layers respectively associated with one of the one or more object maps. The object layers include image data illustrative of the at least one object and one or more trace effects at least partially attributable to the at least one object such that the one or more object layers and the background layer can be re-combined with modified relative timings.
Public/Granted literature
- US20230206955A1 Re-Timing Objects in Video Via Layered Neural Rendering Public/Granted day:2023-06-29
Information query