公开(公告)号：US10565734B2

公开(公告)日：2020-02-18

申请号：US15451831

申请日：2017-03-07

Applicant: GOOGLE LLC

Inventor： Brendan Bevensee ,  Tingfang Du ,  Jon Karafin ,  Joel Merritt ,  Duane Petrovich ,  Gareth Spor

IPC: G06T7/80 ,  H04N5/225 ,  G06T7/557 ,  G06T5/50 ,  H04N17/00 ,  G02B6/08 ,  H04N13/282 ,  G06T7/13 ,  G06T7/521 ,  G01S17/08 ,  G01S17/02 ,  G01S7/481

Abstract: An image capture system includes a plurality of image sensors arranged in a pattern such that gaps exist between adjacent image sensors of the plurality of image sensors. Each of the image sensors may be configured to capture sensor image data. The image capture system may also have a main lens configured to direct incoming light along an optical path, a microlens array positioned within the optical path, and a plurality of tapered fiber optic bundles. Each tapered fiber optic bundle may have a leading end positioned within the optical path, and a trailing end positioned proximate one of the image sensors. The leading end may have a larger cross-sectional area than the trailing end. Sensor data from the image sensors may be combined to generate a single light-field image that is substantially unaffected by the gaps.

公开(公告)号：US10545215B2

公开(公告)日：2020-01-28

申请号：US15703553

申请日：2017-09-13

Applicant: Google LLC

Inventor： Jon Karafin ,  Gang Pan ,  Thomas Nonn ,  Jie Tan

IPC: G01S3/78 ,  H04N5/232 ,  G06K9/62 ,  G06T7/11 ,  H04N13/204 ,  H04N5/225

Abstract: A light-field video stream may be processed to modify the camera pathway from which the light-field video stream is projected. A plurality of target pixels may be selected, in a plurality of key frames of the light-field video stream. The target pixels may be used to generate a camera pathway indicative of motion of the camera during generation of the light-field video stream. The camera pathway may be adjusted to generate an adjusted camera pathway. This may be done, for example, to carry out image stabilization. The light-field video stream may be projected to a viewpoint defined by the adjusted camera pathway to generate a projected video stream with the image stabilization.

公开(公告)号：US20190079158A1

公开(公告)日：2019-03-14

申请号：US15703553

申请日：2017-09-13

Applicant: Google LLC

Inventor： Jon Karafin ,  Gang Pan ,  Thomas Nonn ,  Jie Tan

IPC: G01S3/78 ,  H04N13/02 ,  G06T7/11 ,  G06K9/62 ,  H04N5/232

Abstract: A light-field video stream may be processed to modify the camera pathway from which the light-field video stream is projected. A plurality of target pixels may be selected, in a plurality of key frames of the light-field video stream. The target pixels may be used to generate a camera pathway indicative of motion of the camera during generation of the light-field video stream. The camera pathway may be adjusted to generate an adjusted camera pathway. This may be done, for example, to carry out image stabilization. The light-field video stream may be projected to a viewpoint defined by the adjusted camera pathway to generate a projected video stream with the image stabilization.

公开(公告)号：US11328446B2

公开(公告)日：2022-05-10

申请号：US15635894

申请日：2017-06-28

Applicant: Google LLC

Inventor： Jie Tan ,  Gang Pan ,  Jon Karafin ,  Thomas Nonn ,  Julio C. Hernandez Zaragoza

IPC: G06T7/80 ,  G06T3/00 ,  G06T7/521 ,  G06K9/46 ,  G01S17/08 ,  G01S17/42 ,  H04N13/271 ,  G01S17/86 ,  H04N13/232

Abstract: Depths of one or more objects in a scene may be measured with enhanced accuracy through the use of a light-field camera and a depth sensor. The light-field camera may capture a light-field image of the scene. The depth sensor may capture depth sensor data of the scene. Light-field depth data may be extracted from the light-field image and used, in combination with the sensor depth data, to generate a depth map indicative of distance between the light-field camera and one or more objects in the scene. The depth sensor may be an active depth sensor that transmits electromagnetic energy toward the scene; the electromagnetic energy may be reflected off of the scene and detected by the active depth sensor. The active depth sensor may have a 360° field of view; accordingly, one or more mirrors may be used to direct the electromagnetic energy between the active depth sensor and the scene.