公开(公告)号：US20240378844A1

公开(公告)日：2024-11-14

申请号：US18195401

申请日：2023-05-10

Applicant: Google LLC

Inventor： Lucy YU ,  Andrew LIU ,  Orly LIBA

IPC: G06V10/26 ,  G06T5/00 ,  G06T7/194 ,  G06V10/75 ,  G06V10/774 ,  G06V10/82 ,  G06V20/40

Abstract: A media application derives a bystander mask from an image by analyzing the image with a bystander segmentation model, wherein the image depicts a bystander and the bystander mask identifies a plurality of first pixels in the image that are associated with the bystander. The media derives a shadow mask for the bystander by analyzing the image with a shadow segmentation model, wherein the image and the bystander mask are provided as input to the shadow segmentation model, and wherein the shadow mask identifies a plurality of second pixels in the image that are associated with a shadow of the bystander. The media application modifies the image to update pixel values of the plurality of first pixels and the plurality of second pixels such that the bystander and the shadow are erased from the image.

公开(公告)号：US20240346631A1

公开(公告)日：2024-10-17

申请号：US18293678

申请日：2022-06-30

Applicant: Google LLC

Inventor： Orly LIBA ,  Pedro VELEZ ,  Siyang LI ,  Huizhong CHEN ,  Marcel PUYAT ,  Yanan BAO

IPC: G06T5/77 ,  G06T5/60 ,  G06T7/11 ,  G06T7/136 ,  G06T7/194

CPC classification number: G06T5/77 ,  G06T5/60 ,  G06T7/11 ,  G06T7/136 ,  G06T7/194 ,  G06T2207/20021 ,  G06T2207/20081 ,  G06T2207/30196 ,  G06T2210/12

Abstract: A media application detects a bystander in an initial image. The media application generates a bystander box that includes the bystander, wherein all pixels for the bystander are within the bystander box. The media application generates localizer boxes that encompass the bystander and one or more objects that are attached to the bystander. The media application aggregates the bystander box and one or more of the localizer boxes to form an aggregated box. The media application applies a segmenter to the initial image, based on the aggregated box, to segment the bystander and the one or more objects from the initial image to generate a bystander mask, wherein the bystander mask includes a subset of pixels within the aggregated box. The media application generates an inpainted image that replaces all pixels within the bystander mask with pixels that match a background in the initial image.

公开(公告)号：US20230118460A1

公开(公告)日：2023-04-20

申请号：US17968634

申请日：2022-10-18

Applicant: Google LLC

Inventor： Orly LIBA ,  Nikhil KARNAD ,  Nori KANAZAWA ,  Yael Pritch KNAAN ,  Huizhong CHEN ,  Longqi CAI

IPC: G06V10/26 ,  G06V10/774 ,  G06V10/764 ,  G06V20/20 ,  G06T5/00 ,  G06T11/00 ,  G06T5/20

Abstract: A media application generates training data that includes a first set of media items and a second set of media items, where the first set of media items correspond to the second set of media items and include distracting objects that are manually segmented. The media application trains a segmentation machine-learning model based on the training data to receive a media item with one or more distracting objects and to output a segmentation mask for one or more segmented objects that correspond to the one or more distracting objects.

公开(公告)号：US20230118361A1

公开(公告)日：2023-04-20

申请号：US17968645

申请日：2022-10-18

Applicant: Google LLC

Inventor： Orly LIBA ,  Navin SARMA ,  Yael Pritch KNAAN ,  Alexander SCHIFFHAUER ,  Longqi CAI ,  David JACOBS ,  Huizhong CHEN ,  Siyang LI ,  Bryan FELDMAN

IPC: G06T3/40 ,  G06T7/11 ,  G06T5/00

Abstract: A media application receives user input that indicates one or more objects to be erased from a media item. The media application translates the user input to a bounding box. The media application provides a crop of the media item based on the bounding box to a segmentation machine-learning model. The segmentation machine-learning model outputs a segmentation mask for one or more segmented objects in the crop of the media item and a corresponding segmentation score that indicates a quality of the segmentation mask.

公开(公告)号：US20240394852A1

公开(公告)日：2024-11-28

申请号：US18691569

申请日：2022-08-01

Applicant: Google LLC

Inventor： Orly LIBA ,  Lucy YU ,  Yael Pritch KNAAN

IPC: G06T5/70 ,  G06T7/50 ,  G06V10/25 ,  G06V40/16

Abstract: Implementations described herein relate to methods, computing devices, and non-transitory computer-readable media to generate an output image. In some implementations, a method includes estimating depth for an image to obtain a depth. The method further includes generating a focal table for the image that includes parameters that indicate a focal range and at least one of a front slope or a back slope. The method further includes determining if one or more faces are detected in the image. The method further includes, if one or more faces are detected in the image, identifying a respective face bounding box for each face and adjusting the focal table to include the face bounding boxes. The method further includes, if no faces are detected in the image, scaling the focal table. The method further includes, applying blur to the image using the focal table and the depth map to generate an output image.

公开(公告)号：US20230325985A1

公开(公告)日：2023-10-12

申请号：US18022396

申请日：2021-10-14

Applicant: GOOGLE LLC

Inventor： Soo Ye KIM ,  Orly LIBA ,  Rahul GARG ,  Nori KANAZAWA ,  Neal WADHWA ,  Kfir ABERMAN ,  Huiwen CHANG

IPC: G06T5/00 ,  G06T5/50 ,  G06T3/40

CPC classification number: G06T5/005 ,  G06T5/50 ,  G06T3/4046 ,  G06T2207/20016

Abstract: A method includes receiving an input image. The input image corresponds to one or more masked regions to be inpainted. The method includes providing the input image to a first neural network. The first neural network outputs a first inpainted image at a first resolution, and the one or more masked regions are inpainted in the first inpainted image. The method includes creating a second inpainted image by increasing a resolution of the first inpainted image from the first resolution to a second resolution. The second resolution is greater than the first resolution such that the one or more inpainted masked regions have an increased resolution. The method includes providing the second inpainted image to a second neural network. The second neural network outputs a first refined inpainted image at the second resolution, and the first refined inpainted image is a refined version of the second inpainted image.