公开(公告)号：US20250087244A1

公开(公告)日：2025-03-13

申请号：US18954868

申请日：2024-11-21

Applicant: Fyusion, Inc.

Inventor： Stefan Johannes Josef Holzer ,  Radu Bogdan Rusu ,  Vladimir Roumenov Glavtchev ,  Alexander Jay Bruen Trevor

IPC: G11B27/32 ,  G06F16/68 ,  G11B27/10 ,  H04N13/349

Abstract: According to particular embodiments, one process includes retrieving a multi-view interactive digital media representation that includes numerous images fused together into content and context models. The process next includes retrieving and processing audio data to be integrated into the multi-view interactive digital media representation. A first segment of audio data may be associated with a first position in the multi-view interactive digital media representation. In other examples, a first segment of audio data may be associated with a visual position or the location of a camera in the multi-view interactive digital media representation. The audio data may be played in coordination with the multi-view interactive digital media representation based on a user's navigation through the multi-view interactive digital media representation, where the first segment is played when the first position or first visual position is reached.

公开(公告)号：US20240211511A1

公开(公告)日：2024-06-27

申请号：US18601914

申请日：2024-03-11

Applicant: Fyusion, Inc.

Inventor： Stefan Johannes Josef Holzer ,  Abhishek Kar ,  Alexander Jay Bruen Trevor ,  Pantelis Kalogiros ,  Ioannis Spanos ,  Radu Bogdan Rusu

IPC: G06F16/583 ,  G06V10/10 ,  G06V20/20 ,  G06V20/64

CPC classification number: G06F16/5838 ,  G06V10/17 ,  G06V20/20 ,  G06V20/64

Abstract: Provided are mechanisms and processes for performing visual search using multi-view digital media representations, such as surround views. In one example, a process includes receiving a visual search query that includes a surround view of an object to be searched, where the surround view includes spatial information, scale information, and different viewpoint images of the object. The surround view is compared to stored surround views by comparing spatial information and scale information of the surround view to spatial information and scale information of the stored surround views. A correspondence measure is then generated indicating the degree of similarity between the surround view and a possible match. At least one search result is then transmitted with a corresponding image in response to the visual search query.

公开(公告)号：US11972556B2

公开(公告)日：2024-04-30

申请号：US18068454

申请日：2022-12-19

Applicant: Fyusion, Inc.

Inventor： Stefan Johannes Josef Holzer ,  Abhishek Kar ,  Matteo Munaro ,  Pavel Hanchar ,  Radu Bogdan Rusu

IPC: G06T7/00 ,  G06T3/4038 ,  G06T5/50 ,  G06V10/44 ,  G06V10/80 ,  G06V20/20 ,  G06V20/64 ,  H04N23/45

CPC classification number: G06T7/001 ,  G06T3/4038 ,  G06T5/50 ,  G06T7/0004 ,  G06V10/44 ,  G06V10/454 ,  G06V10/803 ,  G06V20/20 ,  G06V20/647 ,  H04N23/45

Abstract: A background scenery portion may be identified in each of a plurality of image sets of an object, where each image set includes images captured simultaneously from different cameras. A correspondence between the image sets may determined, where the correspondence tracks control points associated with the object and present in multiple images. A multi-view interactive digital media representation of the object that is navigable in one or more dimensions and that includes the image sets may be generated and stored.

公开(公告)号：US20240098233A1

公开(公告)日：2024-03-21

申请号：US18520442

申请日：2023-11-27

Applicant: Fyusion, Inc.

Inventor： Alexander Jay Bruen Trevor ,  Chris Beall ,  Vladimir Glavtchev ,  Stefan Johannes Josef Holzer ,  Radu Bogdan Rusu

IPC: H04N13/117 ,  G01B11/24 ,  G01B21/22 ,  G06T7/00 ,  H04N13/398 ,  H04N23/62 ,  H04N23/698 ,  H04N23/90 ,  H04N25/40

CPC classification number: H04N13/117 ,  G01B11/24 ,  G01B21/22 ,  G06T7/00 ,  H04N13/398 ,  H04N23/62 ,  H04N23/698 ,  H04N23/90 ,  H04N25/41

Abstract: Various embodiments of the present invention relate generally to systems and methods for analyzing and manipulating images and video. In particular, a multi-view interactive digital media representation can be generated from live images captured from a camera. The live images can include an object. An angular view of the object captured in the live images can be estimated using sensor data from an inertial measurement unit. The determined angular views can be used to select from among the live images. The multi-view interactive digital media representation can include a plurality of images where each of the plurality of images includes the object from a different camera view. When the plurality of images is output to a display, the object can appear to undergo a 3-D rotation through the determined angular view where the 3-D rotation of the object is generated without a 3-D polygon model of the object.

公开(公告)号：US11876948B2

公开(公告)日：2024-01-16

申请号：US17814820

申请日：2022-07-25

Applicant: Fyusion, Inc.

Inventor： Alexander Jay Bruen Trevor ,  Chris Beall ,  Vladimir Glavtchev ,  Stefan Johannes Josef Holzer ,  Radu Bogdan Rusu

IPC: H04N13/117 ,  G01B21/22 ,  H04N13/398 ,  G01B11/24 ,  H04N23/62 ,  H04N23/90 ,  H04N23/698 ,  H04N25/40 ,  G06T7/00

CPC classification number: H04N13/117 ,  G01B11/24 ,  G01B21/22 ,  G06T7/00 ,  H04N13/398 ,  H04N23/62 ,  H04N23/698 ,  H04N23/90 ,  H04N25/41

Abstract: Various embodiments of the present invention relate generally to systems and methods for analyzing and manipulating images and video. In particular, a multi-view interactive digital media representation can be generated from live images captured from a camera. The live images can include an object. An angular view of the object captured in the live images can be estimated using sensor data from an inertial measurement unit. The determined angular views can be used to select from among the live images. The multi-view interactive digital media representation can include a plurality of images where each of the plurality of images includes the object from a different camera view. When the plurality of images is output to a display, the object can appear to undergo a 3-D rotation through the determined angular view where the 3-D rotation of the object is generated without a 3-D polygon model of the object.

公开(公告)号：US11783443B2

公开(公告)日：2023-10-10

申请号：US16518558

申请日：2019-07-22

Applicant: Fyusion, Inc.

Inventor： Stefan Johannes Josef Holzer ,  Matteo Munaro ,  Martin Markus Hubert Wawro ,  Abhishek Kar ,  Pavel Hanchar ,  Krunal Ketan Chande ,  Radu Bogdan Rusu

IPC: G06T7/70 ,  G06T3/00

CPC classification number: G06T3/0037 ,  G06T7/70 ,  G06T2207/20044 ,  G06T2207/30244 ,  G06T2207/30248

Abstract: According to various embodiments, component information may be identified for each input image of an object. The component information may indicate a portion of the input image in which a particular component of the object is depicted. A viewpoint may be determined for each input image that indicates a camera pose for the input image relative to the object. A three-dimensional skeleton of the object may be determined based on the viewpoints and the component information. A multi-view panel corresponding to the designated component of the object that is navigable in three dimensions and that the portions of the input images in which the designated component of the object is depicted may be stored on a storage device.

公开(公告)号：US11776142B2

公开(公告)日：2023-10-03

申请号：US17144831

申请日：2021-01-08

Applicant: Fyusion, Inc.

Inventor： Stefan Johannes Josef Holzer ,  Matteo Munaro ,  Radu Bogdan Rusu

IPC: G06T7/38 ,  G06N3/04 ,  G06T19/00 ,  G06T17/20 ,  G06T7/33 ,  G06T7/00 ,  G06T15/20 ,  H04N23/60

CPC classification number: G06T7/38 ,  G06N3/04 ,  G06T7/33 ,  G06T7/97 ,  G06T15/205 ,  G06T17/20 ,  G06T19/006 ,  H04N23/64 ,  G06T2219/004

Abstract: Mappings are determined between viewpoints of an object and an object model representing the object. Each mapping identifies a location on the object model corresponding with a portion of the object captured in one of the viewpoints. Tags for the object model are created based on the mappings, where each tag links one of the viewpoints to one of the locations on the object model. A user interface that includes the object model and the tags is provided for presentation on a display screen in a user interface. One of the viewpoints is presented in the user interface when the corresponding tag is selected in the object model.

公开(公告)号：US20230217001A1

公开(公告)日：2023-07-06

申请号：US18183917

申请日：2023-03-14

Applicant: Fyusion, Inc.

Inventor： Stefan Johannes Josef Holzer ,  Dave Morrison ,  Radu Bogdan Rusu ,  George Haber ,  Keith Martin

IPC: H04N13/221 ,  H04N13/344 ,  H04N13/275 ,  H04N13/178 ,  H04N13/189 ,  H04N13/279 ,  H04N13/243 ,  H04N13/366 ,  G06T7/285 ,  G06T17/00 ,  H04N13/117

CPC classification number: H04N13/221 ,  H04N13/344 ,  H04N13/275 ,  H04N13/178 ,  H04N13/189 ,  H04N13/279 ,  H04N13/243 ,  H04N13/366 ,  G06T7/285 ,  G06T17/00 ,  H04N13/117 ,  G06T2219/2008 ,  G06T2207/10021 ,  G06T2207/30252

Abstract: Various embodiments describe systems and processes for capturing and generating multi-view interactive digital media representations (MIDMRs). In one aspect, a method for automatically generating a MIDMR comprises obtaining a first MIDMR and a second MIDMR. The first MIDMR includes a convex or concave motion capture using a recording device and is a general object MIDMR. The second MIDMR is a specific feature MIDMR. The first and second MIDMRs may be obtained using different capture motions. A third MIDMR is generated from the first and second MIDMRs, and is a combined embedded MIDMR. The combined embedded MIDMR may comprise the second MIDMR being embedded in the first MIDMR, forming an embedded second MIDMR. The third MIDMR may include a general view in which the first MIDMR is displayed for interactive viewing by a user on a user device. The embedded second MIDMR may not be viewable in the general view.

公开(公告)号：US11138432B2

公开(公告)日：2021-10-05

申请号：US15604938

申请日：2017-05-25

Applicant: Fyusion, Inc.

Inventor： Stefan Johannes Josef Holzer ,  Stephen David Miller ,  Pantelis Kalogiros ,  Radu Bogdan Rusu

IPC: G06K9/00 ,  G06F16/583 ,  G06F16/587 ,  G06K9/62

Abstract: Provided are mechanisms and processes for visual feature tagging in multi-view interactive digital media representations (MIDMRs). In one example, a process includes receiving a visual feature tagging request that includes an MIDMR of an object to be searched, where the MIDMR includes spatial information, scale information, and different viewpoint images of the object. A visual feature in the MIDMR is identified, and visual feature correspondence information is created that links information identifying the visual feature with locations in the viewpoint images. At least one image associated with the MIDMR is transmitted in response to the feature tagging request.

公开(公告)号：US20210225065A1

公开(公告)日：2021-07-22

申请号：US17144866

申请日：2021-01-08

Applicant: Fyusion, Inc.

Inventor： Stefan Johannes Josef Holzer ,  Julius Santiago ,  Milos Vlaski ,  Endre Ajandi ,  Radu Bogdan Rusu

IPC: G06T15/20 ,  G06T13/20 ,  H04N13/282

Abstract: A three-dimensional representation of a scene captured in an action shot base video may be determined. The three-dimensional representation may identify a camera pose. A representation of an object may be determined from a multi-view representation of the object that includes images of the object and that is navigable in one or more dimensions. An action shot video of the scene that includes a rendering of the object determined based on the representation and the camera pose may be generated.