Abstract:
The present disclosure is directed to collaborative feature learning using social media data. For example, a machine learning system may identify social media data that includes user behavioral data, which indicates user interactions with content item. Using the identified social user behavioral data, the machine learning system may determine latent representations from the content items. In some embodiments, the machine learning system may train a machine-learning model based on the latent representations. Further, the machine learning system may extract features of the content item from the trained machine-learning model.
Abstract:
Robust techniques for self-calibration of a moving camera observing a planar scene. Plane-based self-calibration techniques may take as input the homographies between images estimated from point correspondences and provide an estimate of the focal lengths of all the cameras. A plane-based self-calibration technique may be based on the enumeration of the inherently bounded space of the focal lengths. Each sample of the search space defines a plane in the 3D space and in turn produces a tentative Euclidean reconstruction of all the cameras that is then scored. The sample with the best score is chosen and the final focal lengths and camera motions are computed. Variations on this technique handle both constant focal length cases and varying focal length cases.
Abstract:
Image classification techniques using images with separate grayscale and color channels are described. In one or more implementations, an image classification network includes grayscale filters and color filters which are separate from the grayscale filters. The grayscale filters are configured to extract grayscale features from a grayscale channel of an image, and the color filters are configured to extract color features from a color channel of the image. The extracted grayscale features and color features are used to identify an object in the image, and the image is classified based on the identified object.
Abstract:
Patch partition and image processing techniques are described. In one or more implementations, a system includes one or more modules implemented at least partially in hardware. The one or more modules are configured to perform operations including grouping a plurality of patches taken from a plurality of training samples of images into respective ones of a plurality of partitions, calculating an image processing operator for each of the partitions, determining distances between the plurality of partitions that describe image similarity of patches of the plurality of partitions, one to another, and configuring a database to provide the determined distance and the image processing operator to process an image in response to identification of a respective partition that corresponds to a patch taken from the image.
Abstract:
An initialization technique is described for determining and reconstructing a set of initial keyframes covering a portion of an image sequence according to point trajectories that may, for example, be used in an adaptive reconstruction algorithm implemented by a structure from motion (SFM) technique. A goal of the initialization technique is to compute an initial reconstruction from a subset of frames in the image sequence. Two initial keyframes are selected from a set of temporally spaced keyframe candidates, the two initial keyframes are reconstructed, and then one or more additional keyframes between the two initial keyframes are selected and reconstructed. Output of the initialization technique is a set of initial keyframes and the initial reconstruction.
Abstract:
In embodiments of optical flow accounting for image haze, digital images may include objects that are at least partially obscured by a haze that is visible in the digital images, and an estimate of light that is contributed by the haze in the digital images can be determined. The haze can be cleared from the digital images based on the estimate of the light that is contributed by the haze, and clearer digital images can be generated. An optical flow between the clearer digital images can then be computed, and the clearer digital images refined based on the optical flow to further clear the haze from the images in an iterative process to improve visibility of the objects in the digital images.
Abstract:
Image editing techniques are disclosed that support a number of physically-based image editing tasks, including object insertion and relighting. The techniques can be implemented, for example in an image editing application that is executable on a computing system. In one such embodiment, the editing application is configured to compute a scene from a single image, by automatically estimating dense depth and diffuse reflectance, which respectively form the geometry and surface materials of the scene. Sources of illumination are then inferred, conditioned on the estimated scene geometry and surface materials and without any user input, to form a complete 3D physical scene model corresponding to the image. The scene model may include estimates of the geometry, illumination, and material properties represented in the scene, and various camera parameters. Using this scene model, objects can be readily inserted and composited into the input image with realistic lighting, shadowing, and perspective.
Abstract:
A system may be configured as an image recognition machine that utilizes an image feature representation called local feature embedding (LFE). LFE enables generation of a feature vector that captures salient visual properties of an image to address both the fine-grained aspects and the coarse-grained aspects of recognizing a visual pattern depicted in the image. Configured to utilize image feature vectors with LFE, the system may implement a nearest class mean (NCM) classifier, as well as a scalable recognition algorithm with metric learning and max margin template selection. Accordingly, the system may be updated to accommodate new classes with very little added computational cost. This may have the effect of enabling the system to readily handle open-ended image classification problems.
Abstract:
An initialization technique that may, for example, be used in an adaptive reconstruction algorithm implemented by structure from motion (SFM) techniques. The initialization technique computes an initial reconstruction from a subset of frames in an image sequence. The initialization technique may be performed to determine and reconstruct a set of initial keyframes covering a portion of the image sequence according to the point trajectories. In the initialization technique, a set of temporally spaced keyframe candidates is determined and two initial keyframes are selected from the set of keyframe candidates. The two initial keyframes are reconstructed, and then one or more additional keyframes between the two initial keyframes are selected and reconstructed.
Abstract:
A technique for selecting a particular reconstruction technique to be applied to an image sequence. The technique may analyze an input image sequence and, based on one or more characteristics of the image sequence, select a reconstruction technique as the appropriate technique for the image sequence from among a set of reconstruction techniques. For example, the set may include two or more of a rotation-based reconstruction technique, a plane-based reconstruction technique, and a general 3D reconstruction technique. The selection technique may be combined with the reconstruction techniques to produce a system that takes as input an image sequence or a set of point trajectories, selects an appropriate reconstruction technique, and applies the selected reconstruction technique to generate an estimate of camera motion and camera intrinsic parameters for the image sequence. The technique may be adapted to select among other types of techniques that may be applied to image sequences.