公开(公告)号：US20230262259A1

公开(公告)日：2023-08-17

申请号：US17670978

申请日：2022-02-14

Applicant: Microsoft Technology Licensing, LLC

Inventor： Luming LIANG ,  Zhicheng GENG ,  Ilya Dmitriyevich ZHARKOV ,  Tianyu DING

IPC: H04N19/587 ,  H04N19/59 ,  H04N19/61 ,  H04N19/132

CPC classification number: H04N19/587 ,  H04N19/59 ,  H04N19/61 ,  H04N19/132

Abstract: A technique is described herein for temporally and spatially interpolating input video information, to produce output video information having a higher frame rate and a higher resolution compared to that exhibited by the input video information. The technique generates feature information based on plural frames of the input video information. The technique then produces the output video information based on the feature information using an architecture having, in order, a multi-stage encoding operation, a query-generating operation, and a multi-stage decoding operation. Each encoding stage produces an instance of encoder attention information that expresses identified relations across the plural frames of the input video information. Each decoding stage operates on an instance of encoder attention information produced by a corresponding encoding stage. The transformer architecture is compact and is capable of interpolating the input video information in real time.

公开(公告)号：US20220122235A1

公开(公告)日：2022-04-21

申请号：US17073256

申请日：2020-10-16

Applicant: Microsoft Technology Licensing, LLC

Inventor： Luming LIANG ,  Ilya Dmitriyevich ZHARKOV ,  Vivek PRADEEP ,  Faezeh AMJADI

IPC: G06T5/00 ,  G06T5/50 ,  G06T3/40 ,  G06N3/08 ,  G06N3/04

Abstract: A computational photography system is described herein including a guidance system and a detail enhancement system. The guidance system uses a first neural network that maps an original image provided by an image sensor to a guidance image, which represents a color-corrected and lighting-corrected version of the original image. A combination unit combines the original image and the guidance image to produce a combined image. A detail-enhancement system then uses a second neural network to map the combined image to a predicted image. The predicted image supplements the guidance provided by the first neural network by sharpening details in the original image. A training system is also described herein for training the first and second neural networks. The training system alternates in the data it feeds the second neural network, first using a guidance image as input to the second neural network, and then using a corresponding ground-truth image.

公开(公告)号：US20220400226A1

公开(公告)日：2022-12-15

申请号：US17347481

申请日：2021-06-14

Applicant: Microsoft Technology Licensing, LLC

Inventor： Luming LIANG ,  Tianyu DING ,  Ilya Dmitriyevich ZHARKOV

IPC: H04N7/01 ,  G06N3/04 ,  G06K9/46 ,  H04N19/587

Abstract: Systems and methods for generating interpolated images are disclosed. In examples, image features are extracted from a first image and a second image; such image features may be warped using first and second plurality of parameters. A first candidate intermediate frame may be generated based on the warped first features and the warped second features. Multi-scale features associated with the image features extracted from the first image and the second image may be obtained and warped using the first and second plurality of parameters. A second candidate intermediate frame may be generated based on the warped first multi-scale features and the warped second multi-scale features. By blending the first candidate intermedia frame with the second candidate intermediate frame, an interpolated image may be generated.

公开(公告)号：US20240403643A1

公开(公告)日：2024-12-05

申请号：US18325379

申请日：2023-05-30

Applicant: Microsoft Technology Licensing, LLC

Inventor： Tianyi CHEN ,  Tianyu DING ,  Luming LIANG ,  Ilya Dmitriyevich ZHARKOV

IPC: G06N3/082 ,  G06N3/0464

Abstract: Technologies described herein relate to training and compressing a computer-implemented model. To that end, an untrained computer-implemented model is obtained, where the untrained computer-implemented model is to be trained and compressed. The untrained computer-implemented model includes an operator that comprises a structure. Further, training data is obtained, where the training data is to be employed to train the computer-implemented model. Upon receipt of a request from a user, the untrained computer-implemented model is trained and compressed based upon the training data. The untrained computer-implemented model is trained and compressed without further input from the user, such that a trained and compressed computer-implemented model is generated. The trained and compressed model does not include the structure.

公开(公告)号：US20230343136A1

公开(公告)日：2023-10-26

申请号：US17729987

申请日：2022-04-26

Applicant: Microsoft Technology Licensing, LLC

Inventor： Yatao ZHONG ,  Faezeh AMJADI ,  Ilya Dmitriyevich ZHARKOV

IPC: G06V40/16 ,  G06T3/00 ,  G06V20/59 ,  G06V10/82 ,  G06V10/44 ,  H04N21/4788

CPC classification number: G06V40/168 ,  G06T3/0093 ,  G06V20/597 ,  G06V10/82 ,  G06V10/454 ,  H04N21/4788

Abstract: A face-processing system is described for producing a target image based on a source image and driving information. The source image includes data depicting at least a face of a source subject having a source identity, a source pose, and a source expression. The driving information specifies one or more driving characteristics. The target image combines characteristics of the source image and the driving information. According to illustrative implementations, the face-processing system produces the target image by using plural warping subcomponents that operate at plural respective levels of a neural network and at increasing respective resolutions. Each warping subcomponent operates, in part, based on geometric displacement field (GDF) information that describes differences between a source mesh derived from the source image and a driving mesh derived from the driving information.

公开(公告)号：US20250166126A1

公开(公告)日：2025-05-22

申请号：US18513081

申请日：2023-11-17

Applicant: Microsoft Technology Licensing, LLC

Inventor： Tianyu DING ,  Jinxin ZHOU ,  Tianyi CHEN ,  Ilya Dmitriyevich ZHARKOV ,  Luming LIANG

IPC: G06T3/40 ,  G06T5/00

Abstract: The technology described herein provides an improved training framework for a diffusion model used for a super resolution (SR) task. In particular, the technology provides diffusion rectification to correct a training-sampling discrepancy inherent in current training methods. The technology also provides estimation-adaptation. The diffusion rectification portion of the technology uses an estimated HR image, rather than a ground truth HR image as the seed to the forward process. This improves model performance issues caused by a training-sampling discrepancy. The training-sampling discrepancy occurs because the training and sampling processes do not use the same data. The estimation adaption strategy injects ground truth to the plurality of noisy images to reduce the training-estimation error in the images. In an aspect, a different amount of ground truth is injected into training images based on the training image's location in the Markov chain.

公开(公告)号：US20240428431A1

公开(公告)日：2024-12-26

申请号：US18339211

申请日：2023-06-21

Applicant: Microsoft Technology Licensing, LLC

Inventor： Tianyi CHEN ,  Tianyu DING ,  Luming LIANG ,  Ilya Dmitriyevich ZHARKOV

IPC: G06T7/50 ,  G06T7/13 ,  G06T7/60

Abstract: A method performed by a processor of a computing system is described herein, where the method includes obtaining an image that includes an object having a shape, where a boundary of the shape of the object in the digital image is labeled in the digital image. The method also includes computing an encoding for the shape, where computing the encoding for the shape includes partitioning the shape into multiple partitions. Computing the encoding for the shape further includes, for the multiple partitions, computing angle-based contour descriptors that represent boundaries of the partitions, where the encoding for the shape of the object is based upon the angle-based contour descriptors.

公开(公告)号：US20230412785A1

公开(公告)日：2023-12-21

申请号：US17843545

申请日：2022-06-17

Applicant: Microsoft Technology Licensing, LLC

Inventor： Karim Henrik BENCHEMSI ,  Aleksandar UZELAC ,  Ilya Dmitriyevich ZHARKOV

IPC: H04N13/128 ,  H04N13/302 ,  G06T7/194 ,  G06T7/70 ,  G06V40/16

CPC classification number: H04N13/128 ,  H04N13/302 ,  G06T2207/30201 ,  G06T7/70 ,  G06V40/161 ,  G06T7/194

Abstract: Technologies disclosed herein relate to construction of a composite image to provide a parallax effect. An image is generated, and a first computer-readable image and a second computer-readable image are generated based upon the image. The first computer-readable image represents foreground of a scene and the second computer-readable image represents background of a scene. A location of eyes of a viewer relative to a display is determined, and the first computer-readable image is overlaid upon the second computer-readable image at a position that is based upon the location of the eyes of the viewer.