公开(公告)号：US10091405B2

公开(公告)日：2018-10-02

申请号：US15438542

申请日：2017-02-21

Applicant: FotoNation Cayman Limited

Inventor： Gabriel Molina

IPC: H04N5/225 ,  H04N5/232 ,  H04N5/238 ,  H04N5/235 ,  H04N5/272 ,  G06T3/40 ,  G06T5/50 ,  G06T7/55 ,  H04N5/265

Abstract: Systems and methods for reducing motion blur in images or video in ultra low light with array cameras in accordance with embodiments of the invention are disclosed. In one embodiment, a method for synthesizing an image from multiple images captured using an array camera includes capturing image data using active cameras within an array camera, where the active cameras are configured to capture image data and the image data includes pixel brightness values that form alternate view images captured from different viewpoints, determining sets of corresponding pixels in the alternate view images where each pixel in a set of corresponding pixels is chosen from a different alternate view image, summing the pixel brightness values for corresponding pixels to create pixel brightness sums for pixel locations in an output image, and synthesizing an output image from the viewpoint of the output image using the pixel brightness sums.

公开(公告)号：US20180211402A1

公开(公告)日：2018-07-26

申请号：US15858974

申请日：2017-12-29

Applicant: FotoNation Cayman Limited

Inventor： Florian Ciurea ,  Kartik Venkataraman ,  Gabriel Molina ,  Dan Lelescu

IPC: G06T7/593 ,  G06T7/557 ,  G02B27/00 ,  G06T15/20 ,  H04N13/00 ,  H04N9/097 ,  G06T7/80

CPC classification number: G06T7/593 ,  G02B27/0075 ,  G06T7/557 ,  G06T7/85 ,  G06T15/20 ,  G06T2200/21 ,  G06T2207/10012 ,  G06T2207/10024 ,  G06T2207/10052 ,  H04N9/097 ,  H04N13/128 ,  H04N13/232 ,  H04N13/243 ,  H04N2013/0081 ,  H04N2013/0088

Abstract: Systems in accordance with embodiments of the invention can perform parallax detection and correction in images captured using array cameras. Due to the different viewpoints of the cameras, parallax results in variations in the position of objects within the captured images of the scene. Methods in accordance with embodiments of the invention provide an accurate account of the pixel disparity due to parallax between the different cameras in the array, so that appropriate scene-dependent geometric shifts can be applied to the pixels of the captured images when performing super-resolution processing. In a number of embodiments, generating depth estimates considers the similarity of pixels in multiple spectral channels. In certain embodiments, generating depth estimates involves generating a confidence map indicating the reliability of depth estimates.

公开(公告)号：US10027901B2

公开(公告)日：2018-07-17

申请号：US15651877

申请日：2017-07-17

Applicant: FotoNation Cayman Limited

Inventor： Kartik Venkataraman ,  Amandeep S. Jabbi ,  Robert H. Mullis

IPC: H04N5/225 ,  H04N5/247 ,  H04N5/357 ,  H04N5/265 ,  H04N5/262 ,  H04N5/232 ,  H04N5/33 ,  H04N5/228 ,  G06T3/40 ,  H04N5/355 ,  H04N5/353 ,  H04N9/04 ,  H04N5/369 ,  H04N3/14 ,  H04N1/195 ,  H01L27/146 ,  G06T7/55 ,  G06T7/70 ,  G02B3/00 ,  G02B5/20 ,  G02B13/00 ,  G02B27/00 ,  G06T7/557 ,  H04N13/02 ,  H04N5/335 ,  G06T1/20 ,  G02B27/12 ,  G02B27/10 ,  H04N5/376 ,  H04N9/083 ,  H04N13/00

CPC classification number: H04N5/247 ,  G02B3/0006 ,  G02B5/20 ,  G02B5/208 ,  G02B13/0015 ,  G02B13/0085 ,  G02B27/0025 ,  G02B27/1066 ,  G02B27/123 ,  G06T1/20 ,  G06T3/4053 ,  G06T7/50 ,  G06T7/55 ,  G06T7/557 ,  G06T7/70 ,  G06T2207/10052 ,  G06T2207/30244 ,  H01L27/14618 ,  H01L27/14621 ,  H01L27/14625 ,  H01L27/14632 ,  H01L27/14685 ,  H01L27/14687 ,  H01L2924/0002 ,  H04N1/195 ,  H04N1/19589 ,  H04N1/19594 ,  H04N3/1593 ,  H04N5/2253 ,  H04N5/2254 ,  H04N5/2258 ,  H04N5/228 ,  H04N5/23212 ,  H04N5/23229 ,  H04N5/23232 ,  H04N5/23238 ,  H04N5/23296 ,  H04N5/262 ,  H04N5/2621 ,  H04N5/265 ,  H04N5/33 ,  H04N5/335 ,  H04N5/3415 ,  H04N5/3537 ,  H04N5/35545 ,  H04N5/357 ,  H04N5/3575 ,  H04N5/3696 ,  H04N5/376 ,  H04N9/04 ,  H04N9/045 ,  H04N13/128 ,  H04N13/243 ,  H04N13/257 ,  H04N2013/0081 ,  H01L2924/00

Abstract: A camera array, an imaging device and/or a method for capturing image that employ a plurality of imagers fabricated on a substrate is provided. Each imager includes a plurality of pixels. The plurality of imagers include a first imager having a first imaging characteristics and a second imager having a second imaging characteristics. The images generated by the plurality of imagers are processed to obtain an enhanced image compared to images captured by the imagers. Each imager may be associated with an optical element fabricated using a wafer level optics (WLO) technology.

公开(公告)号：US20180109782A1

公开(公告)日：2018-04-19

申请号：US15797126

申请日：2017-10-30

Applicant: FotoNation Cayman Limited

Inventor： Jacques Duparre ,  Andrew Kenneth John McMahon ,  Dan Lelescu ,  Kartik Venkataraman ,  Gabriel Molina

IPC: H04N17/00 ,  H04N5/225 ,  H04N5/367 ,  H04N5/217

CPC classification number: H04N17/002 ,  H04N5/2171 ,  H04N5/2254 ,  H04N5/367

Abstract: Systems and methods for detecting defective camera arrays, optic arrays and/or sensors are described. One embodiment includes capturing image data using a camera array; dividing the captured images into a plurality of corresponding image regions; identifying the presence of localized defects in any of the cameras by evaluating the image regions in the captured images; and detecting a defective camera array using the image processing system when the number of localized defects in a specific set of image regions exceeds a predetermined threshold, where the specific set of image regions is formed by: a common corresponding image region from at least a subset of the captured images; and any additional image region in a given image that contains at least one pixel located within a predetermined maximum parallax shift distance along an epipolar line from a pixel within said common corresponding image region within the given image.

公开(公告)号：US20180048830A1

公开(公告)日：2018-02-15

申请号：US15687882

申请日：2017-08-28

Applicant: FotoNation Cayman Limited

Inventor： Kartik Venkataraman ,  Amandeep S. Jabbi ,  Robert H. Mullis ,  Jacques Duparre ,  Shane Ching-Feng Hu

IPC: H04N5/247 ,  H04N5/365 ,  H04N13/02 ,  G02B3/00 ,  G02B13/00 ,  G02B5/20

CPC classification number: G02B13/0015 ,  G02B3/0056 ,  G02B3/0062 ,  G02B5/20 ,  G02B5/201 ,  G06T7/0051 ,  G06T7/557 ,  G06T11/60 ,  G06T19/20 ,  G06T2200/04 ,  G06T2207/10028 ,  G06T2207/10052 ,  G06T2207/20221 ,  H04N5/2253 ,  H04N5/2254 ,  H04N5/2258 ,  H04N5/23232 ,  H04N5/23238 ,  H04N5/23296 ,  H04N5/247 ,  H04N5/262 ,  H04N5/265 ,  H04N5/332 ,  H04N5/3415 ,  H04N5/349 ,  H04N5/35545 ,  H04N5/357 ,  H04N5/365 ,  H04N9/045 ,  H04N9/09 ,  H04N9/097 ,  H04N9/735 ,  H04N13/128 ,  H04N13/239 ,  H04N13/257 ,  H04N2013/0081

Abstract: Systems and methods for implementing array cameras configured to perform super-resolution processing to generate higher resolution super-resolved images using a plurality of captured images and lens stack arrays that can be utilized in array cameras are disclosed. An imaging device in accordance with one embodiment of the invention includes at least one imager array, and each imager in the array comprises a plurality of light sensing elements and a lens stack including at least one lens surface, where the lens stack is configured to form an image on the light sensing elements, control circuitry configured to capture images formed on the light sensing elements of each of the imagers, and a super-resolution processing module configured to generate at least one higher resolution super-resolved image using a plurality of the captured images.

公开(公告)号：US09813616B2

公开(公告)日：2017-11-07

申请号：US14933871

申请日：2015-11-05

Applicant: FotoNation Cayman Limited

Inventor： Dan Lelescu ,  Ankit K. Jain

IPC: H04N5/232 ,  G06T3/40 ,  H04N19/53 ,  H04N19/54 ,  H04N19/59 ,  G06K9/46 ,  G06K9/62 ,  G06T11/60 ,  G06T7/13 ,  G06T7/246

CPC classification number: H04N5/23232 ,  G06K9/4604 ,  G06K9/6201 ,  G06T3/4053 ,  G06T3/4069 ,  G06T7/13 ,  G06T7/246 ,  G06T11/60 ,  G06T2207/20221 ,  H04N19/53 ,  H04N19/54 ,  H04N19/59

Abstract: Systems and methods in accordance with embodiments of the invention enable feature based high resolution motion estimation from low resolution images captured using an array camera. One embodiment includes performing feature detection with respect to a sequence of low resolution images to identify initial locations for a plurality of detected features in the sequence of low resolution images, where the at least one sequence of low resolution images is part of a set of sequences of low resolution images captured from different perspectives. The method also includes synthesizing high resolution image portions, where the synthesized high resolution image portions contain the identified plurality of detected features from the sequence of low resolution images. The method further including performing feature detection within the high resolution image portions to identify high precision locations for the detected features, and estimating camera motion using the high precision locations for said plurality of detected features.

公开(公告)号：US09800859B2

公开(公告)日：2017-10-24

申请号：US14705885

申请日：2015-05-06

Applicant: FotoNation Cayman Limited

Inventor： Kartik Venkataraman ,  Paul Gallagher ,  Ankit Jain ,  Semyon Nisenzon

IPC: H04N13/02 ,  G01P3/38 ,  G06T7/593 ,  G06T7/285 ,  G06T7/557 ,  G06T7/55 ,  G06T7/579

CPC classification number: H04N13/243 ,  G01P3/38 ,  G06T7/285 ,  G06T7/55 ,  G06T7/557 ,  G06T7/579 ,  G06T7/593 ,  G06T2207/10012 ,  G06T2207/10021

Abstract: Systems and methods for stereo imaging with camera arrays in accordance with embodiments of the invention are disclosed. In one embodiment, a method of generating depth information for an object using two or more array cameras that each include a plurality of imagers includes obtaining a first set of image data captured from a first set of viewpoints, identifying an object in the first set of image data, determining a first depth measurement, determining whether the first depth measurement is above a threshold, and when the depth is above the threshold: obtaining a second set of image data of the same scene from a second set of viewpoints located known distances from one viewpoint in the first set of viewpoints, identifying the object in the second set of image data, and determining a second depth measurement using the first set of image data and the second set of image data.

公开(公告)号：US09779319B2

公开(公告)日：2017-10-03

申请号：US14667503

申请日：2015-03-24

Applicant: FotoNation Cayman Limited

Inventor： Kartik Venkataraman ,  Semyon Nisenzon ,  Dan Lelescu

IPC: G06K9/36 ,  G06T9/00 ,  G06T9/20 ,  H04N19/597 ,  H04N13/00 ,  H04N13/02 ,  H04N19/625 ,  H04N19/136 ,  G06T3/40 ,  H04N19/85 ,  G06T15/08 ,  G06T7/50

Abstract: Systems and methods for storing images synthesized from light field image data and metadata describing the images in electronic files in accordance with embodiments of the invention are disclosed. One embodiment includes a processor and memory containing an encoding application and light field image data, where the light field image data comprises a plurality of low resolution images of a scene captured from different viewpoints. In addition, the encoding application configures the processor to synthesize a higher resolution image of the scene from a reference viewpoint using the low resolution images, where synthesizing the higher resolution image involves creating a depth map that specifies depths from the reference viewpoint for pixels in the higher resolution image; encode the higher resolution image; and create a light field image file including the encoded image, the low resolution images, and metadata including the depth map.

公开(公告)号：US09743051B2

公开(公告)日：2017-08-22

申请号：US15012044

申请日：2016-02-01

Applicant: FotoNation Cayman Limited

Inventor： Kartik Venkataraman ,  Paul Gallagher ,  Dan Lelescu ,  Andrew Kenneth John McMahon ,  Jacques Duparre ,  Bedabrata Pain

IPC: H04N5/225 ,  H04N9/09 ,  H04N5/232 ,  H04N5/247 ,  H04N5/33 ,  H04N5/349 ,  H04N5/369 ,  H04N5/378

CPC classification number: H04N9/09 ,  H04N5/2252 ,  H04N5/2254 ,  H04N5/23232 ,  H04N5/247 ,  H04N5/332 ,  H04N5/349 ,  H04N5/3692 ,  H04N5/378

Abstract: Systems and methods in accordance with embodiments of the invention implement modular array cameras using sub-array modules. In one embodiment, an X×Y sub-array module includes: an X×Y arrangement of focal planes, where X and Y are each greater than or equal to 1; and an X×Y arrangement of lens stacks, the X×Y arrangement of lens stacks being disposed relative to the X×Y arrangement of focal planes so as to form an X×Y arrangement of cameras, where each lens stack has a field of view that is shifted with respect to the field-of-views of each other lens stack so that each shift includes a sub-pixel shifted view of the scene; and image data output circuitry that is configured to output image data from the X×Y sub-array module that can be aggregated with image data from other sub-array modules so that an image of the scene can be constructed.

公开(公告)号：US09706132B2

公开(公告)日：2017-07-11

申请号：US14952195

申请日：2015-11-25

Applicant: FotoNation Cayman Limited

Inventor： Semyon Nisenzon ,  Kartik Venkataraman

IPC: H04N5/247 ,  H04N9/097 ,  H04N5/33 ,  G02B3/00 ,  H04N5/225 ,  H04N9/73 ,  H01L27/146 ,  G02B5/20

CPC classification number: H04N5/247 ,  G02B3/0056 ,  G02B3/0062 ,  G02B5/201 ,  H01L27/14621 ,  H01L27/14627 ,  H01L27/14685 ,  H04N5/2254 ,  H04N5/33 ,  H04N9/097 ,  H04N9/735

Abstract: Systems and methods in accordance with embodiments of the invention pattern array camera modules with π filter groups. In one embodiment, an array camera module includes: an M×N imager array including a plurality of focal planes, where each focal plane includes an array of pixels; an M×N optic array of lens stacks, where each lens stack corresponds to a focal plane, and where each lens stack forms an image of a scene on its corresponding focal plane; where each pairing of a lens stack and focal plane thereby defines a camera; where at least one row in the M×N array of cameras includes at least one red camera, one green camera, and one blue camera; and where at least one column in the M×N array of cameras includes at least one red camera, one green camera, and one blue camera.