公开(公告)号：US20240249523A1

公开(公告)日：2024-07-25

申请号：US18560609

申请日：2022-05-11

Applicant: Google LLC

Inventor： Forrester H. Cole ,  Andrew Zisserman ,  Tali Dekel ,  William Tafel Freeman ,  Erika Lu ,  Michael Rubinstein

IPC: G06V20/40 ,  G06T7/194 ,  G06T7/246 ,  G06T7/73 ,  G06V10/26 ,  G06V10/776 ,  G06V10/82

CPC classification number: G06V20/46 ,  G06T7/194 ,  G06T7/246 ,  G06T7/73 ,  G06V10/26 ,  G06V10/776 ,  G06V10/82 ,  G06T2207/10016 ,  G06T2207/10024 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: The present disclosure provides systems and methods for identifying and extracting object-related effects in videos. Given an ordinary video and a rough segmentation mask overtime of one or more subjects of interest, example systems proposed herein can estimate an omnimatte for each subject—an alpha matte and color image that includes the subject along with all its related time-varying scene elements. Example implementations of the proposed models can be trained only on the input video in a self-supervised manner, without any manual labels, and are generic. For example, the models can produce omnimattes automatically for arbitrary objects and a variety of effects.

公开(公告)号：US20230274548A1

公开(公告)日：2023-08-31

申请号：US18008204

申请日：2020-06-10

Applicant: GOOGLE LLC

Inventor： Debidatta Dwibedi ,  Yusuf Aytar ,  Jonathan Tompson ,  Andrew Zisserman ,  Pierre Sermanet

IPC: G06V20/40 ,  G06V10/74 ,  G06V10/82 ,  G06V10/771

CPC classification number: G06V20/48 ,  G06V10/761 ,  G06V10/82 ,  G06V10/771

Abstract: Techniques are disclosed that enable processing a video capturing a periodic activity using a repetition network to generate periodic output (e.g., a period length of the periodic activity captured in the video and/or a frame wise periodicity indication of the video capturing the periodic activity). Various implementations include a class agnostic repetition network which can be used to generate periodic output for a wide variety of periodic activities. Additional or alternative implementations include generating synthetic repetition videos which can be utilized to train the repetition network.

公开(公告)号：US20250166379A1

公开(公告)日：2025-05-22

申请号：US18949777

申请日：2024-11-15

Applicant: Google LLC

Inventor： Alessio Tonioni ,  Bruno Korbar ,  Federico Tombari ,  Andrew Zisserman ,  Yongqin Xian

IPC: G06V20/40 ,  G06F40/35 ,  G06V10/46

Abstract: Methods, systems, and apparatus for video understanding. In one aspect, a conditioned resampler model receives video features of multiple video frames of a video processed by a visual encoder and token embeddings for a specified task. The conditioned resampler model generates conditioned resampler embeddings according to the specified task in response to the video features and token embeddings provided as input. The conditioned resampler embeddings are provided to a large language model as input. The large language model generates, in response to the input conditioned resampler embeddings, a text response to the specified task.