公开(公告)号：US11138432B2

公开(公告)日：2021-10-05

申请号：US15604938

申请日：2017-05-25

Applicant: Fyusion, Inc.

Inventor： Stefan Johannes Josef Holzer ,  Stephen David Miller ,  Pantelis Kalogiros ,  Radu Bogdan Rusu

IPC: G06K9/00 ,  G06F16/583 ,  G06F16/587 ,  G06K9/62

Abstract: Provided are mechanisms and processes for visual feature tagging in multi-view interactive digital media representations (MIDMRs). In one example, a process includes receiving a visual feature tagging request that includes an MIDMR of an object to be searched, where the MIDMR includes spatial information, scale information, and different viewpoint images of the object. A visual feature in the MIDMR is identified, and visual feature correspondence information is created that links information identifying the visual feature with locations in the viewpoint images. At least one image associated with the MIDMR is transmitted in response to the feature tagging request.

公开(公告)号：US11044464B2

公开(公告)日：2021-06-22

申请号：US15607334

申请日：2017-05-26

Applicant: Fyusion, Inc.

Inventor： Stefan Johannes Josef Holzer ,  Stephen David Miller ,  Pantelis Kalogiros ,  George Haber ,  Radu Bogdan Rusu

IPC: H04N13/00 ,  H04N13/395 ,  H04N13/117 ,  G06F16/21 ,  G06F1/16 ,  H04N5/247 ,  G06F3/0484

Abstract: Various embodiments disclosed herein relate to systems and methods for analyzing and manipulating images and video. Methods as disclosed herein may include retrieving, using a processor, a multi-view interactive digital media representation (MIDMR) from a storage location, the MIDMR including a content model and a context model, the content model characterizing an object, and the context model characterizing scenery surrounding the object. The methods may also include receiving, using the processor, at least one dynamic content input associated with the retrieved MIDMR, the dynamic content input being received while a user is interacting with the MIDMR. The methods may further include implementing, using the processor, one or more modifications associated with the MIDMR based, at least in part, on the received at least one dynamic content input, the one or more modifications modifying a presentation and functionality of the MIDMR.

公开(公告)号：US10852902B2

公开(公告)日：2020-12-01

申请号：US16426323

申请日：2019-05-30

Applicant: Fyusion, Inc.

Inventor： Stefan Johannes Josef Holzer ,  Stephen David Miller ,  Radu Bogdan Rusu

IPC: G06F3/0481 ,  G06F16/78 ,  G06K9/22 ,  G06T15/20 ,  G06K9/00 ,  G06F16/58

Abstract: Various embodiments of the present disclosure relate generally to systems and methods for automatic tagging of objects on a multi-view interactive digital media representation of a dynamic entity. According to particular embodiments, the spatial relationship between multiple images and video is analyzed together with location information data, for purposes of creating a representation referred to herein as a multi-view interactive digital media representation for presentation on a device. Multi-view interactive digital media representations correspond to multi-view interactive digital media representations of the dynamic objects in backgrounds. A first multi-view interactive digital media representation of a dynamic object is obtained. Next, the dynamic object is tagged. Then, a second multi-view interactive digital media representation of the dynamic object is generated. Finally, the dynamic object in the second multi-view interactive digital media representation is automatically identified and tagged.

公开(公告)号：US10726593B2

公开(公告)日：2020-07-28

申请号：US14860983

申请日：2015-09-22

Applicant: Fyusion, Inc.

Inventor： Stefan Johannes Josef Holzer ,  Stephen David Miller ,  Martin Saelzle ,  Radu Bogdan Rusu

IPC: G09G5/00 ,  G06T11/60 ,  G06T5/00 ,  H04N13/111

Abstract: Various embodiments of the present invention relate generally to systems and methods for artificially rendering images using viewpoint interpolation and/or extrapolation. According to particular embodiments, a transformation between a first frame and a second frame is estimated, where the first frame includes a first image captured from a first location and the second frame includes a second image captured from a second location. An artificially rendered image corresponding to a third location positioned on a trajectory between the first location and the second location is then generated by interpolating a transformation from the first location to the third location and from the third location to the second location and gathering image information from the first frame and the second frame by transferring first image information from the first frame to the third frame based on the interpolated transformation and second image information from the second frame to the third frame based on the interpolated transformation. The first image information and the second image information are then combined. If an occlusion is created by a change in layer placement between the first frame and second frame, this occlusion is detected and missing data is replaced to fill the occlusion.

公开(公告)号：US10725609B2

公开(公告)日：2020-07-28

申请号：US15724087

申请日：2017-10-03

Applicant: Fyusion, Inc.

Inventor： Stefan Johannes Josef Holzer ,  Stephen David Miller ,  Radu Bogdan Rusu ,  Alexander Jay Bruen Trevor ,  Krunal Ketan Chande

IPC: G06F3/0481 ,  G06F3/01 ,  H04L29/06 ,  G06K9/00 ,  G06T15/20 ,  G06T19/00 ,  H04N21/00 ,  H04N13/111 ,  G06K9/72 ,  H04N13/00 ,  H04N13/261

Abstract: Various embodiments of the present disclosure relate generally to systems and methods for generating multi-view interactive digital media representations in a virtual reality environment. According to particular embodiments, a plurality of images is fused into a first content model and a first context model, both of which include multi-view interactive digital media representations of s. Next, a virtual reality environment is generated using the first content model and the first context model. The virtual reality environment includes a first layer and a second layer. The user can navigate through and within the virtual reality environment to switch between multiple viewpoints of the content model via corresponding physical movements. The first layer includes the first content model and the second layer includes a second content model and wherein selection of the first layer provides access to the second layer with the second content model.

公开(公告)号：US10713851B2

公开(公告)日：2020-07-14

申请号：US16186994

申请日：2018-11-12

Applicant: Fyusion, Inc.

Inventor： Stefan Johannes Josef Holzer ,  Alexander Jay Bruen Trevor ,  Martin Saelzle ,  Stephen David Miller ,  Radu Bogdan Rusu

IPC: G06T19/00 ,  G06T7/70 ,  G06T7/20 ,  G06F3/01 ,  G06T11/00 ,  G06T7/55 ,  G06T7/33 ,  G06T7/246

Abstract: Various embodiments of the present invention relate generally to systems and methods for analyzing and manipulating images and video. According to particular embodiments, the spatial relationship between multiple images and video is analyzed together with location information data, for purposes of creating a representation referred to herein as a surround view for presentation on a device. A real object can be tracked in the live image data for the purposes of creating a surround view using a number of tracking points. As a camera is moved around the real object, virtual objects can be rendered into live image data to create synthetic images where a position of the tracking points can be used to position the virtual object in the synthetic image. The synthetic images can be output in real-time. Further, virtual objects in the synthetic images can be incorporated into surround views.

公开(公告)号：US10521954B2

公开(公告)日：2019-12-31

申请号：US16237516

申请日：2018-12-31

Applicant: Fyusion, Inc.

Inventor： Stefan Johannes Josef Holzer ,  Stephen David Miller ,  Radu Bogdan Rusu

IPC: G06T15/00 ,  G06T15/20 ,  G06T17/00 ,  H04N13/279 ,  H04N13/282 ,  G06F16/532 ,  G06F16/738 ,  G06F16/583 ,  H04N5/232 ,  G06F3/0484 ,  G06T3/40 ,  H04N5/265

Abstract: Various embodiments of the present invention relate generally to systems and methods for analyzing and manipulating images and video. According to particular embodiments, the spatial relationship between multiple images and video is analyzed together with location information data, for purposes of creating a representation referred to herein as a surround view. In particular embodiments, a surround view can be generated by combining a panoramic view of an object with a panoramic view of a distant scene, such that the object panorama is placed in a foreground position relative to the distant scene panorama. Such combined panoramas can enhance the interactive and immersive viewing experience of the surround view.

公开(公告)号：US20190392650A1

公开(公告)日：2019-12-26

申请号：US16564598

申请日：2019-09-09

Applicant: Fyusion, Inc.

Inventor： Stefan Johannes Josef Holzer ,  Alexander Jay Bruen Trevor ,  Michelle Jung-Ah Ho ,  David Klein ,  Stephen David Miller ,  Shuichi Tsutsumi ,  Radu Bogdan Rusu

IPC: G06T19/00 ,  G06K9/00

Abstract: Various embodiments of the present invention relate generally to systems and methods for analyzing and manipulating images and video. According to particular embodiments, the spatial relationship between multiple images and video is analyzed together with location information data, for purposes of creating a representation referred to herein as a surround view for presentation on a device. A visual guide can provided for capturing the multiple images used in the surround view. The visual guide can be a synthetic object that is rendered in real-time into the images output to a display of an image capture device. The visual guide can help user keep the image capture device moving along a desired trajectory.

公开(公告)号：US10469768B2

公开(公告)日：2019-11-05

申请号：US15936141

申请日：2018-03-26

Applicant: Fyusion, Inc.

Inventor： Stefan Johannes Josef Holzer ,  Radu Bogdan Rusu ,  Stephen David Miller ,  Pantelis Kalogiros ,  Matteo Munaro ,  Alexander Jay Bruen Trevor ,  Nico Gregor Sebastian Blodow ,  Luo Yi Tan ,  Mike Penz ,  Martin Markus Hubert Wawro ,  Matthias Reso ,  Chris Beall ,  Yusuke Tomoto ,  Krunal Ketan Chande

IPC: H04N5/272 ,  H04N5/265 ,  G06T19/00 ,  H04N13/111 ,  G06T13/40 ,  G06T7/11 ,  H04N13/282 ,  G06K9/00 ,  H04N5/232 ,  G06T7/194 ,  H04N13/279

Abstract: Various embodiments of the present invention relate generally to systems and methods for analyzing and manipulating images and video. In particular, a multi-view interactive digital media representation (MVIDMR) of a person can be generated from live images of a person captured from a hand-held camera. Using the image data from the live images, a skeleton of the person and a boundary between the person and a background can be determined from different viewing angles and across multiple images. Using the skeleton and the boundary data, effects can be added to the person, such as wings. The effects can change from image to image to account for the different viewing angles of the person captured in each image.

公开(公告)号：US10440351B2

公开(公告)日：2019-10-08

申请号：US15449511

申请日：2017-03-03

Applicant: Fyusion, Inc.

Inventor： Stefan Johannes Josef Holzer ,  Radu Bogdan Rusu ,  Stephen David Miller ,  Pantelis Kalogiros ,  George Haber

IPC: H04N13/279 ,  G06F3/0481 ,  G06F3/0485 ,  G06F1/16 ,  H04N13/243 ,  H04N13/117

Abstract: Various embodiments of the present invention relate generally to systems and methods for analyzing and manipulating images and video. According to particular embodiments, the spatial relationship between multiple images and video is analyzed together with location information data, for purposes of creating a representation referred to herein as a multi-view interactive digital media representation for presentation on a device. Once a multi-view interactive digital media representation is generated, a user can provide navigational inputs, such via tilting of the device, which alter the presentation state of the multi-view interactive digital media representation. The navigational inputs can be analyzed to determine metrics which indicate a user's interest in the multi-view interactive digital media representation.