公开(公告)号：US11405544B2

公开(公告)日：2022-08-02

申请号：US17181799

申请日：2021-02-22

Applicant: SONY GROUP CORPORATION

Inventor： Nikolaos Georgis ,  Gary Lyons ,  Edward Theodore Winter

IPC: H04N5/232 ,  H04N5/247 ,  G06T7/80 ,  H04R1/40 ,  G06T17/00 ,  H04N5/225 ,  H04R3/00

Abstract: An electronic device for programmable rig control for three-dimensional (3D) reconstruction is provided. The electronic device controls a first image sensor to capture a first image of a first scene which includes a set of subjects. The electronic device further feed a first input to a neural network. The electronic device further receives a first output from the neural network based on the fed first input. The electronic device further selects one or more image sensors based on the received first output. The electronic device further controls a first set of structures associated with the selected one or more image sensors to re-arrange a rig around a three-dimensional (3D) physical space. The electronic device further controls a first set of image sensors, in the re-arranged rig, to capture one or more images for generation of one or more three-dimensional (3D) models of a first subject in the 3D physical space.

公开(公告)号：US11200919B2

公开(公告)日：2021-12-14

申请号：US16814056

申请日：2020-03-10

Applicant: Sony Group Corporation

Inventor： Gary Lyons ,  Kenji Suzuki ,  Brent Faust ,  Kevin Barnes ,  Jong Hwa Lee

IPC: G06F3/048 ,  G11B27/34 ,  G06F3/0484 ,  G06K9/00

Abstract: Implementations generally provide a user interface for video annotation tools. In some implementations, a method includes obtaining at least one video of at least one object performing at least one action displaying one or more portions of the at least one video in a user interface. The method further includes displaying a plurality of annotation tracks in the user interface, where each annotation track of the plurality of annotation tracks is associated with one or more of the at least one object and the at least one action in the at least one video. The method further includes obtaining one or more annotations associated with the at least one video based on the plurality of annotation tracks.

公开(公告)号：US11776204B2

公开(公告)日：2023-10-03

申请号：US17154827

申请日：2021-01-21

Applicant: SONY GROUP CORPORATION

Inventor： Jong Hwa Lee ,  Seunghan Kim ,  Gary Lyons

IPC: G06T17/00 ,  G06N3/08 ,  G06T7/529 ,  G06T19/20 ,  G06V40/16 ,  G06T17/10 ,  G06F18/214 ,  G06V10/764 ,  G06V10/774 ,  G06V10/82 ,  G06V10/54 ,  G06V10/44 ,  G06V20/64 ,  G06V40/10

CPC classification number: G06T17/00 ,  G06F18/214 ,  G06N3/08 ,  G06T7/529 ,  G06T17/10 ,  G06T19/20 ,  G06V10/44 ,  G06V10/54 ,  G06V10/764 ,  G06V10/774 ,  G06V10/82 ,  G06V20/647 ,  G06V40/10 ,  G06V40/171

Abstract: An electronic device receives a set of 2D images comprising at least first 2D image of an object of interest and detects a plurality of 2D landmarks on the first 2D image. The detected plurality of 2D landmarks corresponds to shape-features of the object of interest. The electronic device aligns a first three-dimensional (3D) shape model of a reference object by fitting 2D landmarks of the first 3D shape model to the detected plurality of 2D landmarks. The electronic device estimates texture mapping information between the object of interest and the aligned first 3D shape model and generates a dataset by including the first 2D image, the aligned first 3D shape model, and the estimated texture mapping information as a first sample of the dataset. Based on the generated dataset, the electronic device trains a neural network model on a task of 3D reconstruction from a single 2D image.

公开(公告)号：US11748943B2

公开(公告)日：2023-09-05

申请号：US17170739

申请日：2021-02-08

Applicant: SONY GROUP CORPORATION

Inventor： Jong Hwa Lee ,  Seunghan Kim ,  Gary Lyons

IPC: G06K9/62 ,  G06T17/00 ,  G06T7/73 ,  G06T15/04 ,  G06T7/50 ,  G06N3/08 ,  G06N3/04 ,  G06F18/214 ,  G06V10/75 ,  G06V10/74 ,  G06V10/774 ,  G06V10/82 ,  G06V20/64 ,  G06V40/16

CPC classification number: G06T17/00 ,  G06F18/214 ,  G06N3/04 ,  G06N3/08 ,  G06T7/50 ,  G06T7/73 ,  G06T15/04 ,  G06V10/757 ,  G06V10/761 ,  G06V10/774 ,  G06V10/82 ,  G06V20/647 ,  G06V40/171 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/30201

Abstract: An electronic device and method of dataset cleaning is provided. The electronic device receives a dataset comprising a plurality of samples, of which a first sample comprises a 2D image of an object of interest and a 3D shape model of the object of interest. The electronic device determines 2D landmarks from the 2D image and extracts 3D landmarks from the 3D shape model. The electronic device computes an error between the determined 2D landmarks and corresponding 2D locations of the extracted 3D landmarks on the 2D image, based on an error metric. Thereafter, the electronic device determines the computed error to be above a threshold. Based on the determination that the computed error is above the threshold, the electronic device updates the dataset by a removal of the first sample from the dataset and trains a neural network on a task of 3D reconstruction, based on the updated dataset.

公开(公告)号：US11410347B2

公开(公告)日：2022-08-09

申请号：US17196030

申请日：2021-03-09

Applicant: SONY GROUP CORPORATION

Inventor： Jong Hwa Lee ,  Moises Tafolla ,  Praggya Garg ,  Allison Langley ,  Gary Lyons ,  Alexander Stoneham ,  Seunghan Kim ,  Luis Vasconcelos

IPC: G06T11/00 ,  G06F3/048 ,  G06F3/0484

Abstract: A system and method for node-based image colorization is provided. The system receives a first user input via a Graphical User Interface (GUI) of an image/video editing application and controls a display device to display a node graph on the GUI based on the first user input. The node graph includes an input node to select grayscale images and a colorization node which may represent a workflow for colorization of at least a first object in the grayscale images. The system receives a second user input associated with a setting of the colorization node and selects an image colorization model based on the second user input. The system executes the workflow associated with the colorization node to transmit the grayscale images to a computing device that hosts the selected image colorization model, and to receive colorized images including at least the first object colorized based on a color effect.