公开(公告)号：US20200236296A1

公开(公告)日：2020-07-23

申请号：US16692219

申请日：2019-11-22

Applicant: Fyusion, Inc.

Inventor： Stefan Johannes Josef Holzer ,  Abhishek Kar ,  Matteo Munaro ,  Pavel Hanchar ,  Radu Bogdan Rusu

IPC: H04N5/232 ,  G06T7/00 ,  G06T7/70 ,  G06F9/451 ,  G06F17/18

Abstract: One or more images of an object, each from a respective viewpoint, may be captured at a camera at a mobile computing device. The images may be compared to reference data to identify a difference between the images and the reference data. Image capture guidance may be provided on a display screen for capturing another one or more images of the object that includes the identified difference.

公开(公告)号：US20200234397A1

公开(公告)日：2020-07-23

申请号：US16518501

申请日：2019-07-22

Applicant: Fyusion, Inc.

Inventor： Stefan Johannes Josef Holzer ,  Matteo Munaro ,  Aidas Liaudanskas ,  Abhishek Kar ,  Krunal Ketan Chande ,  Radu Bogdan Rusu

IPC: G06T3/00 ,  G06T7/00

Abstract: A three-dimensional (3D) skeleton may be determined based on a plurality of vertices and a plurality of faces in a two-dimensional (2D) mesh in a top-down image of an object. A correspondence mapping between a designated perspective view image and the top-down object image may be determined based on the 3D skeleton. The correspondence mapping may link a respective first location in the top-down object image to a respective second location in the designated perspective view image for each of a plurality of points in the designated perspective view image. A top-down mapped image of the object may be created by determining a first respective pixel value for each of the first locations, with each first respective pixel value being determined based on a second respective pixel value for the respective second location linked with the respective first location via the correspondence mapping.

公开(公告)号：US20200228774A1

公开(公告)日：2020-07-16

申请号：US16574639

申请日：2019-09-18

Applicant: Fyusion, Inc.

Inventor： Abhishek Kar ,  Rodrigo Ortiz Cayon ,  Ben Mildenhall ,  Stefan Johannes Josef Holzer ,  Radu Bogdan Rusu

IPC: H04N13/111 ,  G06T7/557 ,  H04N13/271 ,  G06T7/579 ,  G06T7/70 ,  G06T7/514 ,  G06T15/20

Abstract: An estimated camera pose may be determined for each of a plurality of single plane images of a designated three-dimensional scene. The sampling density of the single plane images may be below the Nyquist rate. However, the sampling density of the single plane images may be sufficiently high such that the single plane images is sufficiently high such that they may be promoted to multiplane images and used to generate novel viewpoints in a light field reconstruction framework. Scene depth information identifying for each of a respective plurality of pixels in the single plane image a respective depth value may be determined for each single plane image. A respective multiplane image including a respective plurality of depth planes may be determined for each single plane image. Each of the depth planes may include a respective plurality of pixels from the respective single plane image.

公开(公告)号：US20200167570A1

公开(公告)日：2020-05-28

申请号：US16778981

申请日：2020-01-31

Applicant: Fyusion, Inc.

Inventor： Chris Beall ,  Abhishek Kar ,  Stefan Johannes Josef Holzer ,  Radu Bogdan Rusu ,  Pavel Hanchar

IPC: G06K9/00 ,  G06T19/00 ,  G06T17/30

Abstract: A multi-view interactive digital media representation (MVIDMR) of an object can be generated from live images of an object captured from a camera. Selectable tags can be placed at locations on the object in the MVIDMR. When the selectable tags are selected, media content can be output which shows details of the object at location where the selectable tag is placed. A machine learning algorithm can be used to automatically recognize landmarks on the object in the frames of the MVIDMR and a structure from motion calculation can be used to determine 3-D positions associated with the landmarks. A 3-D skeleton associated with the object can be assembled from the 3-D positions and projected into the frames associated with the MVIDMR. The 3-D skeleton can be used to determine the selectable tag locations in the frames of the MVIDMR of the object.

公开(公告)号：US10353946B2

公开(公告)日：2019-07-16

申请号：US15409497

申请日：2017-01-18

Applicant: Fyusion, Inc.

Inventor： Stefan Johannes Josef Holzer ,  Abhishek Kar ,  Pantelis Kalogiros ,  Ioannis Spanos ,  Luke Parham ,  Radu Bogdan Rusu

IPC: G06F16/00 ,  G06F16/532 ,  G06F16/583 ,  G06F16/58 ,  G06K9/00 ,  G06K9/22

Abstract: Provided are mechanisms and processes for performing live search using multi-view digital media representations. In one example, a process includes receiving a visual search query from a device for an object to be searched, where the visual search query includes a first set of viewpoints of the object obtained during capture of a first surround view of the object during a live search session. Next, additional recommended viewpoints of the object are identified for the device to capture, where the additional recommended viewpoints are chosen to provide more information about the object. A first set of search results based on the first set of viewpoints and additional recommended viewpoints of the object are transmitted to the device. In response, a second set of viewpoints of the object captured using image capture capabilities of the device are received. A second set of search results with enhanced matches for the object based on the first and second sets of viewpoints are then transmitted to the device. This process may continue iteratively until a desired set of search results is obtained.

公开(公告)号：US10070154B2

公开(公告)日：2018-09-04

申请号：US15427027

申请日：2017-02-07

Applicant: Fyusion, Inc.

Inventor： Stefan Johannes Josef Holzer ,  Matteo Munaro ,  Abhishek Kar ,  Alexander Jay Bruen Trevor ,  Krunal Ketan Chande ,  Radu Bogdan Rusu

IPC: H04N7/173 ,  H04N21/2187 ,  H04N21/4223 ,  H04N21/234

CPC classification number: H04N21/2187 ,  H04N21/23418 ,  H04N21/4223 ,  H04N21/4402 ,  H04N21/6547

Abstract: Provided are mechanisms and processes for performing live filtering in a camera view via client-server communication. In one example, a first video frame in a raw video stream is transmitted from a client device to a server. The client device receives a filter processing message associated with the first video frame that includes filter data for applying a filter to the first video frame. A processor at the client device creates a filtered video stream by applying the filter to a second video frame that occurs in the video stream later than the first video frame.

公开(公告)号：US20180203880A1

公开(公告)日：2018-07-19

申请号：US15409497

申请日：2017-01-18

Applicant: Fyusion, Inc.

Inventor： Stefan Johannes Josef Holzer ,  Abhishek Kar ,  Pantelis Kalogiros ,  Ioannis Spanos ,  Luke Parham ,  Radu Bogdan Rusu

IPC: G06F17/30 ,  G06K9/62 ,  G06K9/72

CPC classification number: G06F16/532 ,  G06F16/583 ,  G06F16/5866 ,  G06K9/00664 ,  G06K9/00979 ,  G06K9/22

Abstract: Provided are mechanisms and processes for performing live search using multi-view digital media representations. In one example, a process includes receiving a visual search query from a device for an object to be searched, where the visual search query includes a first set of viewpoints of the object obtained during capture of a first surround view of the object during a live search session. Next, additional recommended viewpoints of the object are identified for the device to capture, where the additional recommended viewpoints are chosen to provide more information about the object. A first set of search results based on the first set of viewpoints and additional recommended viewpoints of the object are transmitted to the device. In response, a second set of viewpoints of the object captured using image capture capabilities of the device are received. A second set of search results with enhanced matches for the object based on the first and second sets of viewpoints are then transmitted to the device. This process may continue iteratively until a desired set of search results is obtained.

公开(公告)号：US20170148223A1

公开(公告)日：2017-05-25

申请号：US15428104

申请日：2017-02-08

Applicant: Fyusion, Inc.

Inventor： Stefan Johannes Josef HOLZER ,  Yuheng Ren ,  Abhishek Kar ,  Alexander Jay Bruen Trevor ,  Krunal Ketan Chande ,  Martin Josef Nikolaus Saelzle ,  Radu Bogdan Rusu

IPC: G06T19/00 ,  G06T7/174 ,  H04N13/02 ,  G06T19/20 ,  G06T7/11

Abstract: Various embodiments describe systems and processes for generating AR/VR content. In one aspect, a method for generating a three-dimensional (3D) projection of an object is provided. A sequence of images along a camera translation may be obtained using a single lens camera. Each image contains at least a portion of overlapping subject matter, which includes the object. The object is semantically segmented from the sequence of images using a trained neural network to form a sequence of segmented object images, which are then refined using fine-grained segmentation. On-the-fly interpolation parameters are computed and stereoscopic pairs are generated for points along the camera translation from the refined sequence of segmented object images for displaying the object as a 3D projection in a virtual reality or augmented reality environment. Segmented image indices are then mapped to a rotation range for display in the virtual reality or augmented reality environment.

公开(公告)号：US12203872B2

公开(公告)日：2025-01-21

申请号：US17351124

申请日：2021-06-17

Applicant: Fyusion, Inc.

Inventor： Stefan Johannes Josef Holzer ,  Santiago Arano Perez ,  Abhishek Kar ,  Matteo Munaro ,  Pavel Hanchar ,  Radu Bogdan Rusu ,  Martin Markus Hubert Wawro ,  Ashley Wakefield ,  Rodrigo Ortiz-Cayon ,  Josh Faust ,  Jai Chaudhry ,  Nico Gregor Sebastian Blodow ,  Mike Penz

IPC: G06T7/70 ,  G01N21/95 ,  G06T7/00 ,  G06T15/20 ,  G06T19/00

Abstract: Images of an object may be captured at a computing device. Each of the images may be captured from a respective viewpoint based on image capture configuration information identifying one or more parameter values. A multiview image digital media representation of the object may be generated that includes some or all of the images of the object and that is navigable in one or more dimensions.

公开(公告)号：US11967162B2

公开(公告)日：2024-04-23

申请号：US17935239

申请日：2022-09-26

Applicant: Fyusion, Inc.

Inventor： Chris Beall ,  Abhishek Kar ,  Stefan Johannes Josef Holzer ,  Radu Bogdan Rusu ,  Pavel Hanchar

IPC: G06V20/70 ,  G06T17/30 ,  G06T19/00 ,  G06V10/422 ,  G06V10/772 ,  G06V20/10 ,  G06V20/20 ,  G06V20/64

CPC classification number: G06V20/70 ,  G06T17/30 ,  G06T19/003 ,  G06V10/422 ,  G06V10/772 ,  G06V20/10 ,  G06V20/20 ,  G06V20/64

Abstract: A multi-view interactive digital media representation (MVIDMR) of an object can be generated from live images of an object captured from a camera. Selectable tags can be placed at locations on the object in the MVIDMR. When the selectable tags are selected, media content can be output which shows details of the object at location where the selectable tag is placed. A machine learning algorithm can be used to automatically recognize landmarks on the object in the frames of the MVIDMR and a structure from motion calculation can be used to determine 3-D positions associated with the landmarks. A 3-D skeleton associated with the object can be assembled from the 3-D positions and projected into the frames associated with the MVIDMR. The 3-D skeleton can be used to determine the selectable tag locations in the frames of the MVIDMR of the object.