Abstract:
Techniques are disclosed for managing image capture and processing in a multi-camera imaging system. In such a system, a pair of cameras each may output a sequence of frames representing captured image data. The cameras' output may be synchronized to each other to cause synchronism in the image capture operations of the cameras. The system may assess image quality of frames output from the cameras and, based on the image quality, designate a pair of the frames to serve as a “reference frame pair.” Thus, one frame from the first camera and a paired frame from the second camera will be designated as the reference frame pair. The system may adjust each reference frame in the pair using other frames from their respective cameras. The reference frames also may be processed by other operations within the system, such as image fusion.
Abstract:
Techniques to capture and fuse short- and long-exposure images of a scene from a stabilized image capture device are disclosed. More particularly, the disclosed techniques use not only individual pixel differences between co-captured short- and long-exposure images, but also the spatial structure of occluded regions in the long-exposure images (e.g., areas of the long-exposure image(s) exhibiting blur due to scene object motion). A novel device used to represent this feature of the long-exposure image is a “spatial difference map.” Spatial difference maps may be used to identify pixels in the short- and long-exposure images for fusion and, in one embodiment, may be used to identify pixels from the short-exposure image(s) to filter post-fusion so as to reduce visual discontinuities in the output image.
Abstract:
Lens flare mitigation techniques determine which pixels in images of a sequence of images are likely to be pixels affected by lens flare. Once the lens flare areas of the images are determined, unwanted lens flare effects may be mitigated by various approaches, including reducing border artifacts along a seam between successive images, discarding entire images of the sequence that contain lens flare areas, and using tone-mapping to reduce the visibility of lens flare.
Abstract:
Foreign lighting effects, such as lens flare, are very common in natural images. In a two-camera-system, the two images may be fused together to generate one image of a better quality. However, there are frequently different foreign light patterns in the two images that form the image pair, e.g., due to the difference in lens design, sensor and position, etc. Directly fusing such pairs of images will result in non-photorealistic images, with composed foreign light patterns from both images from the image pair. This disclosure describes a general foreign light mitigation scheme to detect all kinds of foreign light region mismatches. The detected foreign light mismatch regions may be deemphasized or excluded in the fusion step, in order to create a fused image that keeps a natural-looking foreign light pattern that is close to what was seen by the user of an image capture device during an image capture preview mode.
Abstract:
Systems, methods, and computer readable media to improve image stabilization operations are described. A novel approach to pixel-based registration of non-reference images to a reference frame in a set of commonly captured images is disclosed which makes use of pyramid decomposition to more efficiently detect corners. The disclosed pixel-based registration operation may also be combined with motion sensor data-based registration approaches to register non-reference images with respect to the reference frame. When the registered non-reference images are combined with the pre-selected reference image, the resulting image is a quality stabilized image.
Abstract:
Systems, methods, and computer readable media to capture and process high dynamic range (HDR) images when appropriate for a scene are disclosed. When appropriate, multiple images at a single—slightly underexposed—exposure value are captured (making a constant bracket HDR capture sequence) and local tone mapping (LTM) applied to each image. Local tone map and histogram information can be used to generate a noise-amplification mask which can be used during fusion operations. Images obtained and fused in the disclosed manner provide high dynamic range with improved noise and de-ghosting characteristics.
Abstract:
Techniques for registering images based on an identified region of interest (ROI) are described. In general, the disclosed techniques identify a region of ROI within an image and assign areas within the image corresponding to those regions more importance during the registration process. More particularly, the disclosed techniques may employ user-input or image content information to identify the ROI. Once identified, features within the ROI may be given more weight or significance during registration operations than other areas of the image having high-feature content but which are not as important to the individual capturing the image.
Abstract:
Special blend operations for wide area-of-view image generation utilizing a “floating auto exposure” scheme are described. Pixel values in the two images being stitched together are blended within a transition band around a “seam.” identified in the overlap region between the images after changes in exposure and/or color saturation are accounted for. In some embodiments, changes in exposure and/or color saturation are accounted for through the use of one or more exposure mapping curves, the selection and use of which are based, at least in part, on a determined “Exposure Ratio” value, i.e., the amount that the camera's exposure settings have deviated from their initial capture settings. In other embodiments, the Exposure Ratio value is also used to determine regions along the seam where either: alpha blending, Poisson blending—or a combination of the two—should be used to blend in the transitional areas on each side of the seam.
Abstract:
Electronic devices, methods, and program storage devices for leveraging machine learning to perform high-resolution and low latency image fusion and/or noise reduction are disclosed. An incoming image stream may be obtained from an image capture device, wherein the incoming image stream comprises a variety of differently-exposed captures, e.g., EV0 images, EV− images, EV+ images, long exposure images, EV0/EV− image pairs, etc., which are received according to a particular pattern. When a capture request is received, two or more intermediate assets may be generated from images from the incoming image stream and fed into a neural network that has been trained to fuse and/or noise reduce the intermediate assets. In some embodiments, the resultant fused image generated from the two or more intermediate assets may have a higher resolution than at least one of the images that were used to generate at least one of the two or more intermediate assets.
Abstract:
Techniques are disclosed for managing image capture and processing in a multi-camera imaging system. In such a system, a pair of cameras each may output a sequence of frames representing captured image data. The cameras' output may be synchronized to each other to cause synchronism in the image capture operations of the cameras. The system may assess image quality of frames output from the cameras and, based on the image quality, designate a pair of the frames to serve as a “reference frame pair.” Thus, one frame from the first camera and a paired frame from the second camera will be designated as the reference frame pair. The system may adjust each reference frame in the pair using other frames from their respective cameras. The reference frames also may be processed by other operations within the system, such as image fusion.