公开(公告)号：US12170755B2

公开(公告)日：2024-12-17

申请号：US17347694

申请日：2021-06-15

Applicant: SONY GROUP CORPORATION

Inventor： Nikolaos Georgis ,  James Kuch ,  Kiyoharu Sassa

IPC: H04N13/398 ,  H04N21/43 ,  G01P15/00

Abstract: A method and system for synchronization of image data is provided. A plurality of image-capture devices is controlled to acquire a plurality of video clips of at least one object. Each video clip of the plurality of video clips is acquired by a corresponding image-capture device of the plurality of image-capture devices in a moving state. From the plurality of image-capture devices, a set of sensor data is acquired. Such data corresponds to the plurality of image-capture devices and is associated with a movement of the plurality of image-capture devices. Thereafter, relative offsets between the set of sensor data are determined by using cross-correlation and matching frames in each of the plurality of video clips is further determined, based on the relative offsets.

公开(公告)号：US20240196084A1

公开(公告)日：2024-06-13

申请号：US18079423

申请日：2022-12-12

Applicant: Sony Group Corporation

Inventor： Luis Vasconcelos ,  Nikolaos Georgis

IPC: H04N23/63 ,  G06T7/70 ,  G06V10/46 ,  H04N13/106

CPC classification number: H04N23/635 ,  G06T7/70 ,  G06V10/462 ,  H04N13/106 ,  H04N2013/0096

Abstract: An apparatus includes one or more processors and logic encoded in one or more non-transitory media for execution by the one or more processors and when executed operable to receive a first image of an object, wherein the first image was captured by a first camera in a set of cameras. The logic is further operable to determine first keypoints and a first pose for the object based on the first image. The logic is further operable to receive a second image from a second camera in the set of cameras. The logic is further operable to determine an overlap of the first image and the second image based on locations of the first keypoints, the second keypoints, the first pose, and the second pose. The logic is further operable to determine a synchronization error between the first image and the second image based on the overlap.

公开(公告)号：US11671551B2

公开(公告)日：2023-06-06

申请号：US17328920

申请日：2021-05-24

Applicant: SONY GROUP CORPORATION

Inventor： Jason Vandekieft ,  Nikolaos Georgis ,  Chad Goolbis

IPC: H04N5/04 ,  G10L25/18 ,  G10L25/51 ,  G06F18/22 ,  H04N23/80 ,  H04N23/90

CPC classification number: H04N5/04 ,  G06F18/22 ,  G10L25/18 ,  G10L25/51 ,  H04N23/80 ,  H04N23/90

Abstract: A method and system for synchronization of image data is provided. From a plurality of image-capture devices, image data including a plurality of image sequences is received. Each image sequence corresponds to one of the plurality of image-capture devices. A set of sensor data is further received from the plurality of image-capture devices. Each sensor data comprises at least one of an Inertial Measurement Unit (IMU) data and audio data. Based on the received set of sensor data, a match is determined between a first set of image frames of a first image sequence and a second set of image frames of a second image sequence. An offset is computed between the first set of image frames and the second set of image frames, based on the match and the first image sequence is synchronized with the second image sequence based on the computed offset.

公开(公告)号：US11405544B2

公开(公告)日：2022-08-02

申请号：US17181799

申请日：2021-02-22

Applicant: SONY GROUP CORPORATION

Inventor： Nikolaos Georgis ,  Gary Lyons ,  Edward Theodore Winter

IPC: H04N5/232 ,  H04N5/247 ,  G06T7/80 ,  H04R1/40 ,  G06T17/00 ,  H04N5/225 ,  H04R3/00

Abstract: An electronic device for programmable rig control for three-dimensional (3D) reconstruction is provided. The electronic device controls a first image sensor to capture a first image of a first scene which includes a set of subjects. The electronic device further feed a first input to a neural network. The electronic device further receives a first output from the neural network based on the fed first input. The electronic device further selects one or more image sensors based on the received first output. The electronic device further controls a first set of structures associated with the selected one or more image sensors to re-arrange a rig around a three-dimensional (3D) physical space. The electronic device further controls a first set of image sensors, in the re-arranged rig, to capture one or more images for generation of one or more three-dimensional (3D) models of a first subject in the 3D physical space.

公开(公告)号：US20240080432A1

公开(公告)日：2024-03-07

申请号：US17901731

申请日：2022-09-01

Applicant: Sony Group Corporation

Inventor： Luis Vanconcelos ,  Nikolaos Georgis

IPC: H04N13/282 ,  G06V10/44 ,  G06V20/50 ,  H04N5/232

CPC classification number: H04N13/282 ,  G06V10/443 ,  G06V20/50 ,  H04N5/232939

Abstract: An apparatus includes one or more processors and logic encoded in one or more non-transitory media for execution by the one or more processors and when executed operable to receive images from a set of cameras that are positioned in a plurality of levels around one or more objects, wherein the set of cameras are positioned at a same orientation and each image is captured by a rolling shutter and associated with a position and orientation of a corresponding camera from the set of cameras. The logic is further operable to, for each level of cameras generate a three-dimensional pointcloud based on the position and orientation of corresponding cameras and matched extracted features. The logic is further operable to merge the three-dimensional pointcloud for each level of the cameras to form a three-dimensional volumetric model of one or more objects.

公开(公告)号：US11727651B2

公开(公告)日：2023-08-15

申请号：US17235255

申请日：2021-04-20

Applicant: SONY GROUP CORPORATION

Inventor： Nikolaos Georgis

IPC: G06T19/00 ,  H03M7/30 ,  G06T7/62 ,  G06T9/00

CPC classification number: G06T19/006 ,  G06T7/62 ,  G06T9/001 ,  H03M7/3059 ,  G06T2207/10028

Abstract: A hybrid implementation enables sharing the processing of 3D data locally and remotely based on processing and bandwidth factors. The hybrid implementation is flexible in determining what information to process locally, what information to transmit to a remote system, and what information to process remotely. Based on the available bandwidth, computing power/availability locally and computing power/availability remotely, the hybrid implementation is able to direct the processing of the data. By performing some of the processing locally and some of the processing remotely, more efficient processing is able to be implemented.

公开(公告)号：US12177558B2

公开(公告)日：2024-12-24

申请号：US18079423

申请日：2022-12-12

Applicant: Sony Group Corporation

Inventor： Luis Vasconcelos ,  Nikolaos Georgis

IPC: H04N23/63 ,  G06T7/70 ,  G06V10/46 ,  H04N13/00 ,  H04N13/106

Abstract: An apparatus includes one or more processors and logic encoded in one or more non-transitory media for execution by the one or more processors and when executed operable to receive a first image of an object, wherein the first image was captured by a first camera in a set of cameras. The logic is further operable to determine first keypoints and a first pose for the object based on the first image. The logic is further operable to receive a second image from a second camera in the set of cameras. The logic is further operable to determine an overlap of the first image and the second image based on locations of the first keypoints, the second keypoints, the first pose, and the second pose. The logic is further operable to determine a synchronization error between the first image and the second image based on the overlap.

公开(公告)号：US12003695B2

公开(公告)日：2024-06-04

申请号：US17901731

申请日：2022-09-01

Applicant: Sony Group Corporation

Inventor： Luis Vanconcelos ,  Nikolaos Georgis

IPC: H04N13/282 ,  G06V10/44 ,  G06V20/50 ,  H04N23/63

CPC classification number: H04N13/282 ,  G06V10/443 ,  G06V20/50 ,  H04N23/633

Abstract: An apparatus includes one or more processors and logic encoded in one or more non-transitory media for execution by the one or more processors and when executed operable to receive images from a set of cameras that are positioned in a plurality of levels around one or more objects, wherein the set of cameras are positioned at a same orientation and each image is captured by a rolling shutter and associated with a position and orientation of a corresponding camera from the set of cameras. The logic is further operable to, for each level of cameras generate a three-dimensional pointcloud based on the position and orientation of corresponding cameras and matched extracted features. The logic is further operable to merge the three-dimensional pointcloud for each level of the cameras to form a three-dimensional volumetric model of one or more objects.

公开(公告)号：US20230099605A1

公开(公告)日：2023-03-30

申请号：US17487290

申请日：2021-09-28

Applicant: SONY GROUP CORPORATION

Inventor： Nikolaos Georgis ,  Kiyoharu Sassa

IPC: G06T17/00 ,  H04L29/06 ,  G06T9/00 ,  G06T15/20 ,  H04N5/247

Abstract: A minimal volumetric 3D transmission implementation enables efficient transmission of a 3D model to a client device. A volumetric 3D model is generated using a camera rig to capture frames of a subject. A viewer is able to select a view of the subject. A system determines an optimal subset of cameras of the camera rig to utilize to capture frames to generate the volumetric 3D model based on the viewer's selected view. The volumetric 3D model is transmitted to the user device. If the user changes the view, the process repeats, and a new subset of cameras are selected to generate the volumetric 3D model at a different angle.

公开(公告)号：US11417125B2

公开(公告)日：2022-08-16

申请号：US17107058

申请日：2020-11-30

Applicant: SONY GROUP CORPORATION

Inventor： Nikolaos Georgis ,  James Kuch

IPC: G06K9/00 ,  G06V20/62 ,  G06V30/413

Abstract: A system and method for recognition of license plate numbers from Bayer-domain image data is provided. The system acquires Bayer-domain image data of a scene comprising a vehicle and inputs such image data to a Deep Neural Network (DNN) model trained for license plate detection (LPD) and license plate recognition (LPR). The system extracts an LPD result as an output of the DNN model. Based on the LPD result, the system extracts, from the input Bayer-domain image data, a region-of-interest (RoI) comprising a license plate image associated with the vehicle. The system inputs the extracted RoI to the DNN model and extracts, as an output of the DNN model, an LPR result which indicates a confidence of the DNN model in recognition of a license plate number of the vehicle from the input RoI. The system determines the license plate number based on the extracted LPR result.