Automatically segmenting images based on natural language phrases

    公开(公告)号:US10089742B1

    公开(公告)日:2018-10-02

    申请号:US15458887

    申请日:2017-03-14

    Abstract: The invention is directed towards segmenting images based on natural language phrases. An image and an n-gram, including a sequence of tokens, are received. An encoding of image features and a sequence of token vectors are generated. A fully convolutional neural network identifies and encodes the image features. A word embedding model generates the token vectors. A recurrent neural network (RNN) iteratively updates a segmentation map based on combinations of the image feature encoding and the token vectors. The segmentation map identifies which pixels are included in an image region referenced by the n-gram. A segmented image is generated based on the segmentation map. The RNN may be a convolutional multimodal RNN. A separate RNN, such as a long short-term memory network, may iteratively update an encoding of semantic features based on the order of tokens. The first RNN may update the segmentation map based on the semantic feature encoding.

    Imaging process initialization techniques

    公开(公告)号:US09911201B2

    公开(公告)日:2018-03-06

    申请号:US15191141

    申请日:2016-06-23

    Inventor: Xin Lu Zhe Lin

    Abstract: Imaging process initialization techniques are described. In an implementation, a color estimate is generated for a plurality of pixels within a region of an image. A plurality of pixels outside of the regions are first identified for each pixel of the plurality of pixels within the region. This may include identification of pixels disposed at opposing directions from the pixel being estimated. A color estimate is determined for each of the plurality of pixels based on the identified pixels. As part of this, a weighting may be employed, such as based on a respective distance of each of the pixels outside of the region to the pixel within the region, a distance along the opposing direction for corresponding pixels outside of the region (e.g., at horizontal or vertical directions), and so forth. The color estimate is then used to initialize an imaging process technique.

    Learned Piece-Wise Patch Regression for Image Enhancement
    23.
    发明申请
    Learned Piece-Wise Patch Regression for Image Enhancement 有权
    学习的片断 - 图像增强的明智的补丁回归

    公开(公告)号:US20140153819A1

    公开(公告)日:2014-06-05

    申请号:US13691190

    申请日:2012-11-30

    CPC classification number: G06T5/002 G06T2207/20081 G06T2207/20084

    Abstract: Systems and methods are provided for providing learned, piece-wise patch regression for image enhancement. In one embodiment, an image manipulation application generates training patch pairs that include training input patches and training output patches. Each training patch pair includes a respective training input patch from a training input image and a respective training output patch from a training output image. The training input image and the training output image include at least some of the same image content. The image manipulation application determines patch-pair functions from at least some of the training patch pairs. Each patch-pair function corresponds to a modification to a respective training input patch to generate a respective training output patch. The image manipulation application receives an input image generates an output image from the input image by applying at least some of the patch-pair functions based on at least some input patches of the input image.

    Abstract translation: 提供了系统和方法,用于为图像增强提供学习的分段补丁回归。 在一个实施例中,图像处理应用产生训练补丁对,其包括训练输入补丁和训练输出补丁。 每个训练补丁对包括来自训练输入图像的相应训练输入补丁和来自训练输出图像的相应训练输出补丁。 训练输入图像和训练输出图像包括至少一些相同的图像内容。 图像处理应用程序从至少一些训练补丁对确定补丁对功能。 每个补丁对功能对应于对相应的训练输入补丁的修改以生成相应的训练输出补丁。 图像处理应用程序接收输入图像,基于输入图像的至少一些输入图像块,通过应用至少一些补丁对功能,从输入图像生成输出图像。

    DEEP HIGH-RESOLUTION STYLE SYNTHESIS
    25.
    发明申请

    公开(公告)号:US20180240257A1

    公开(公告)日:2018-08-23

    申请号:US15438147

    申请日:2017-02-21

    Abstract: In some embodiments, techniques for synthesizing an image style based on a plurality of neural networks are described. A computer system selects a style image based on user input that identifies the style image. The computer system generates an image based on a generator neural network and a loss neural network. The generator neural network outputs the synthesized image based on a noise vector and the style image and is trained based on style features generated from the loss neural network. The loss neural network outputs the style features based on a training image. The training image and the style image have a same resolution. The style features are generated at different resolutions of the training image. The computer system provides the synthesized image to a user device in response to the user input.

    Patch partitions and image processing

    公开(公告)号:US09767540B2

    公开(公告)日:2017-09-19

    申请号:US14280421

    申请日:2014-05-16

    Abstract: Patch partition and image processing techniques are described. In one or more implementations, a system includes one or more modules implemented at least partially in hardware. The one or more modules are configured to perform operations including grouping a plurality of patches taken from a plurality of training samples of images into respective ones of a plurality of partitions, calculating an image processing operator for each of the partitions, determining distances between the plurality of partitions that describe image similarity of patches of the plurality of partitions, one to another, and configuring a database to provide the determined distance and the image processing operator to process an image in response to identification of a respective partition that corresponds to a patch taken from the image.

    Neural Network Image Curation Control
    27.
    发明申请
    Neural Network Image Curation Control 有权
    神经网络图像整形控制

    公开(公告)号:US20160179844A1

    公开(公告)日:2016-06-23

    申请号:US14573963

    申请日:2014-12-17

    Abstract: Neural network image curation techniques are described. In one or more implementations, curation is controlled of images that represent a repository of images. A plurality of images of the repository are curated by one or more computing devices to select representative images of the repository. The curation includes calculating a score based on image and face aesthetics, jointly, for each of the plurality of images through processing by a neural network, ranking the plurality of images based on respective said scores, and selecting one or more of the plurality of images as one of the representative images of the repository based on the ranking and a determination that the one or more said images are not visually similar to images that have already been selected as one of the representative images of the repository.

    Abstract translation: 描述神经网络图像策划技术。 在一个或多个实现中,控制图像的图像的图像库。 存储库的多个图像由一个或多个计算设备进行策划,以选择存储库的代表图像。 该策展包括基于图像和面部美学计算一个分数,通过神经网络的处理来共同地为多个图像中的每个图像,基于相应的分数对多个图像进行排序,并且选择多个图像中的一个或多个 作为基于排名的存储库的代表性图像之一,并且确定一个或多个所述图像在视觉上与已经被选择为存储库的代表图像之一的图像相似。

    Neural Network Patch Aggregation and Statistics
    28.
    发明申请
    Neural Network Patch Aggregation and Statistics 有权
    神经网络补丁和统计

    公开(公告)号:US20160140408A1

    公开(公告)日:2016-05-19

    申请号:US14548170

    申请日:2014-11-19

    CPC classification number: G06K9/4676 G06K9/4628

    Abstract: Neural network patch aggregation and statistical techniques are described. In one or more implementations, patches are generated from an image, e.g., randomly, and used to train a neural network. An aggregation of outputs of patches processed by the neural network may be used to label an image using an image descriptor, such as to label aesthetics of the image, classify the image, and so on. In another example, the patches may be used by the neural network to calculate statistics describing the patches, such as to describe statistics such as minimum, maximum, median, and average of activations of image characteristics of the individual patches. These statistics may also be used to support a variety of functionality, such as to label the image as described above.

    Abstract translation: 描述神经网络补丁聚合和统计技术。 在一个或多个实现中,从图像生成补片,例如随机地,并用于训练神经网络。 由神经网络处理的补丁的输出的聚合可以用于使用图像描述符来标记图像,例如标记图像的美学,对图像进行分类等等。 在另一示例中,神经网络可以使用补丁来计算描述补丁的统计量,例如描述诸如单个补丁的图像特征的激活的最小值,最大值,中值和平均值的统计信息。 这些统计信息也可以用于支持各种功能,例如如上所述标记图像。

    Adaptive denoising with internal and external patches
    29.
    发明授权
    Adaptive denoising with internal and external patches 有权
    自适应去噪内部和外部补丁

    公开(公告)号:US09189834B2

    公开(公告)日:2015-11-17

    申请号:US14080659

    申请日:2013-11-14

    Abstract: In techniques for adaptive denoising with internal and external patches, example image patches taken from example images are grouped into partitions of similar patches, and a partition center patch is determined for each of the partitions. An image denoising technique is applied to image patches of a noisy image to generate modified image patches, and a closest partition center patch to each of the modified image patches is determined. The image patches of the noisy image are then classified as either a common patch or a complex patch of the noisy image, where an image patch is classified based on a distance between the corresponding modified image patch and the closest partition center patch. A denoising operator can be applied to an image patch based on the classification, such as applying respective denoising operators to denoise the image patches that are classified as the common patches of the noisy image.

    Abstract translation: 在使用内部和外部补丁进行自适应去噪的技术中,从示例图像获取的示例图像修补程序分组到类似修补程序的分区中,并为每个分区确定分区中心修补程序。 将图像去噪技术应用于噪声图像的图像补丁以产生修改后的图像斑块,并确定每个修改后的图像斑块的最接近的分割中心斑块。 然后,噪声图像的图像块被分类为噪声图像的公共补丁或复杂补丁,其中基于相应修改的图像补丁和最接近的分割中心补丁之间的距离对图像补丁进行分类。 可以基于分类将去噪算子应用于图像补片,例如应用相应的去噪算子去除被分类为噪声图像的公共斑块的图像斑块。

    Learned piece-wise patch regression for image enhancement
    30.
    发明授权
    Learned piece-wise patch regression for image enhancement 有权
    学习了片面补丁回归图像增强

    公开(公告)号:US09117262B2

    公开(公告)日:2015-08-25

    申请号:US13691190

    申请日:2012-11-30

    CPC classification number: G06T5/002 G06T2207/20081 G06T2207/20084

    Abstract: Systems and methods are provided for providing learned, piece-wise patch regression for image enhancement. In one embodiment, an image manipulation application generates training patch pairs that include training input patches and training output patches. Each training patch pair includes a respective training input patch from a training input image and a respective training output patch from a training output image. The training input image and the training output image include at least some of the same image content. The image manipulation application determines patch-pair functions from at least some of the training patch pairs. Each patch-pair function corresponds to a modification to a respective training input patch to generate a respective training output patch. The image manipulation application receives an input image generates an output image from the input image by applying at least some of the patch-pair functions based on at least some input patches of the input image.

    Abstract translation: 提供了系统和方法,用于为图像增强提供学习的分段补丁回归。 在一个实施例中,图像处理应用产生训练补丁对,其包括训练输入补丁和训练输出补丁。 每个训练补丁对包括来自训练输入图像的相应训练输入补丁和来自训练输出图像的相应训练输出补丁。 训练输入图像和训练输出图像包括至少一些相同的图像内容。 图像处理应用程序从至少一些训练补丁对确定补丁对功能。 每个补丁对功能对应于对相应的训练输入补丁的修改以生成相应的训练输出补丁。 图像处理应用程序接收输入图像,基于输入图像的至少一些输入图像块,通过应用至少一些补丁对功能,从输入图像生成输出图像。

Patent Agency Ranking