公开(公告)号：US10805253B2

公开(公告)日：2020-10-13

申请号：US15396392

申请日：2016-12-30

Applicant: Facebook, Inc.

Inventor： John Samuel Barnett ,  Johannes Peter Kopf

IPC: G06F15/16 ,  H04L12/58 ,  G06F3/01 ,  H04N21/84 ,  H04N21/45 ,  H04N21/4725 ,  G06F16/435 ,  G06Q50/00

Abstract: Systems, methods, and non-transitory computer readable media are configured to detect a concept reflected in a first media content item to which a user is provided access. It is determined that the concept has a threshold level of relevance to the user. The concept is associated with an element that upon selection causes a transition to a second media content item to which the user is provided access, the second media content item reflecting the concept. The element is presented in the first media content item for the user.

公开(公告)号：US20180205941A1

公开(公告)日：2018-07-19

申请号：US15489503

申请日：2017-04-17

Applicant: Facebook, Inc.

Inventor： Johannes Peter Kopf ,  Lars Peter Johannes Hedman ,  Richard Szeliski

IPC: H04N13/02 ,  G06T3/00 ,  G06T3/40 ,  H04N13/00

CPC classification number: H04N13/282 ,  G06T3/0093 ,  G06T3/4038 ,  G06T7/579 ,  G06T2207/10012 ,  G06T2207/10028 ,  H04N13/128 ,  H04N13/15 ,  H04N13/221 ,  H04N13/261 ,  H04N13/271 ,  H04N2013/0081

Abstract: To enable better sharing and preservation of immersive experiences, a graphics system reconstructs a three-dimensional scene from a set of images of the scene taken from different vantage points. The system processes each image to extract depth information therefrom and then stitches the images (both color and depth information) into a multi-layered panorama that includes at least front and back surface layers. The front and back surface layers are then merged to remove redundancies and create connections between neighboring pixels that are likely to represent the same object, while removing connections between neighboring pixels that are not. The resulting layered panorama with depth information can be rendered using a virtual reality (VR) system, a mobile device, or other computing and display platforms using standard rendering techniques, to enable three-dimensional viewing of the scene.

公开(公告)号：US20180190026A1

公开(公告)日：2018-07-05

申请号：US15396382

申请日：2016-12-30

Applicant: Facebook, Inc.

Inventor： John Samuel Barnett ,  Johannes Peter Kopf

IPC: G06T19/00 ,  G06F3/01 ,  G06T11/60 ,  G06K9/00

CPC classification number: G06T11/60 ,  G06F3/011 ,  G06F3/013 ,  G06F3/017 ,  G06F3/04815 ,  G06F3/04817 ,  G06F3/04842 ,  G06K9/00671 ,  G06Q10/10 ,  G06Q50/01

Abstract: Systems, methods, and non-transitory computer readable media are configured to detect a concept reflected in a first media content item to which a user is provided access. It is determined that the concept has a threshold level of relevance to the user. The concept is associated with an element that upon selection causes a transition to a second media content item to which the user is provided access, the second media content item reflecting the concept. The element is presented in the first media content item for the user.

公开(公告)号：US20210366075A1

公开(公告)日：2021-11-25

申请号：US16882322

申请日：2020-05-22

Applicant: Facebook, Inc.

Inventor： Andrei Chtcherbatchenko ,  Francis Yunfeng Ge ,  Bo Yin ,  Shi Chen ,  Fabian Langguth ,  Johannes Peter Kopf ,  Suhib Fakhri Mahmod Alsisan ,  Richard Szeliski

IPC: G06T3/00 ,  G06T7/73 ,  G06T5/50

Abstract: In one embodiment, a method includes generating an outputted sequence of warped images from a captured sequence of images. Using this captured sequence of images, a computing system may determine one or more three-dimensional locations of object features and a corresponding camera position for each image in the captured sequence of images. Utilizing the camera positions for each image, the computing system may determine a view path representing the perspective of a virtual camera. The computing system may identify one or more virtual camera positions for the virtual camera located on the view path, and subsequently warp one or more images from the sequence of captured images to represent the perspective of the virtual camera at each of the respective virtual camera positions. This results in a sequence of warped images that may be outputted for viewing and interaction on a client device.

公开(公告)号：US10789723B1

公开(公告)日：2020-09-29

申请号：US15956177

申请日：2018-04-18

Applicant: Facebook, Inc.

Inventor： Johannes Peter Kopf ,  Brian Dolhansky ,  Suhib Fakhri Mahmod Alsisan

IPC: G06K9/00 ,  G06T7/536 ,  G06N3/08 ,  G06T7/55 ,  G06T17/20 ,  G06N20/00

Abstract: In one embodiment, a method includes generating depth map for a reference image and generating a three-dimensional (3D) model for a plurality of objects in the reference image based on the depth map. The method additionally includes determining, out of the objects in the 3D model, a background object having a boundary adjacent to a foreground object. The method also includes determining that at least a portion of a surface of the background object is hidden by the foreground object and extending, in the 3D model, the surface of the background object to include the portion hidden by the foreground object. The method further includes in-paint pixels of the extended surface of the background object with pixels that approximate the portion of the surface of the background object hidden by the foreground object.

公开(公告)号：US20180189935A1

公开(公告)日：2018-07-05

申请号：US15849379

申请日：2017-12-20

Applicant: Facebook, Inc.

Inventor： Jason George McHugh ,  Michael F. Cohen ,  Johannes Peter Kopf ,  Piotr Dollar

IPC: G06T5/00 ,  G06N99/00 ,  G06T7/13 ,  G06T5/20

CPC classification number: G06T5/002 ,  G06N20/00 ,  G06T5/20 ,  G06T7/11 ,  G06T7/13 ,  G06T7/143 ,  G06T7/194 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/20192

Abstract: Systems, methods, and non-transitory computer-readable media can generate an initial alpha mask for an image based on machine learning techniques. A plurality of uncertain pixels is defined in the initial alpha mask. For each uncertain pixel in the plurality of uncertain pixels, a binary value is assigned based on a nearest certain neighbor determination.

公开(公告)号：US10607318B2

公开(公告)日：2020-03-31

申请号：US15849379

申请日：2017-12-20

Applicant: Facebook, Inc.

Inventor： Jason George McHugh ,  Michael F. Cohen ,  Johannes Peter Kopf ,  Piotr Dollar

IPC: G06T5/00 ,  G06T5/20 ,  G06T7/13 ,  G06T7/143 ,  G06T7/11 ,  G06T7/194 ,  G06N20/00

Abstract: Systems, methods, and non-transitory computer-readable media can generate an initial alpha mask for an image based on machine learning techniques. A plurality of uncertain pixels is defined in the initial alpha mask. For each uncertain pixel in the plurality of uncertain pixels, a binary value is assigned based on a nearest certain neighbor determination.

公开(公告)号：US20180300851A1

公开(公告)日：2018-10-18

申请号：US15953174

申请日：2018-04-13

Applicant: Facebook, Inc.

Inventor： Hadar Elor ,  Michael F. Cohen ,  Johannes Peter Kopf

IPC: G06T3/00 ,  G06K9/00

Abstract: A reactive profile picture brings a profile image to life by displaying short video segments of the target user expressing a relevant emotion in reaction to an action by a viewing user that relates to content associated with the target user in an online system such as a social media web site. The viewing user therefore experiences a real-time reaction in a manner similar to a face-to-face interaction. The reactive profile picture can be automatically generated from either a video input of the target user or from a single input image of the target user.

公开(公告)号：US20180189283A1

公开(公告)日：2018-07-05

申请号：US15396392

申请日：2016-12-30

Applicant: Facebook, Inc.

Inventor： John Samuel Barnett ,  Johannes Peter Kopf

IPC: G06F17/30 ,  H04L29/06 ,  H04L12/58

Abstract: Systems, methods, and non-transitory computer readable media are configured to detect a concept reflected in a first media content item to which a user is provided access. It is determined that the concept has a threshold level of relevance to the user. The concept is associated with an element that upon selection causes a transition to a second media content item to which the user is provided access, the second media content item reflecting the concept. The element is presented in the first media content item for the user.

公开(公告)号：US20180063440A1

公开(公告)日：2018-03-01

申请号：US15247834

申请日：2016-08-25

Applicant: Facebook, Inc.

Inventor： Johannes Peter Kopf

IPC: H04N5/232 ,  G06T7/20 ,  G06T5/00

Abstract: An image processing system generates 360-degree stabilized videos with higher robustness, speed, and smoothing ability using a hybrid 3D-2D stabilization model. The image processing system first receives an input video data (e.g., a 360-degree video data) for rotation stabilization. After tracking feature points through the input video data, the image processing system determines key frames and estimates rotations of key frames using a 3D reasoning based on the tracked feature points. The image processing system also optimizes inner frames between key frames using a 2D analysis based on the estimated key frame rotation. After the 3D reasoning and the 2D analysis, the image processing system may reapply a smoothed version of raw rotations to preserve desirable rotations included in the original input video data, and generates a stabilized version of the input video data (e.g., a 360-degree stabilized video).