公开(公告)号：US20240362891A1

公开(公告)日：2024-10-31

申请号：US18306671

申请日：2023-04-25

Applicant: Microsoft Technology Licensing, LLC

Inventor： Michael BLEYER ,  Pascal PARÉ ,  Paul LEE ,  Aleksander Bogdan BAPST

IPC: G06V10/74 ,  G06V10/25 ,  G06V10/44

CPC classification number: G06V10/761 ,  G06V10/25 ,  G06V10/44

Abstract: A system for selecting motion models for aligning scene content captured by different image sensors, is configurable to (i) access a first image captured by a first image sensor and a second image captured by a second image sensor; (ii) access a set of motion models; (iii) define a reference patch within the second image; (iv) generate a respective match patch for each motion model of the set of motion models; (v) determine a similarity between each respective match patch and the reference patch within the second image; (vi) select a final motion model from the set of motion models based upon the similarity between each respective match patch and the reference patch within the second image; and (vii) utilize the final motion model to generate an output image for display to a user.

公开(公告)号：US20240144526A1

公开(公告)日：2024-05-02

申请号：US17978526

申请日：2022-11-01

Applicant: Microsoft Technology Licensing, LLC

Inventor： Paul LEE ,  Michael BLEYER ,  Christian Markus MAEKELAE

IPC: G06T7/73 ,  G06T7/246 ,  G06T19/00

CPC classification number: G06T7/74 ,  G06T7/251 ,  G06T19/006

Abstract: Techniques for generating an aggregated set of features from multiple images generated by a camera are disclosed. A first image generated by the camera is accessed, where the first image was generated at a first time. A first set of features are identified from within the first image. A second image generated by the camera is accessed, where the second image is generated at a subsequent, second time. A second set of features are identified from within the second image. Movement data is obtained. This movement data details a movement of the camera between the first and second times. The movement data is used to reproject a pose embodied in the first image to correspond to a pose embodied in the second image. The embodiments aggregate the two sets of features to generate the aggregated set of features. The aggregated set of features for the camera are then cached.

公开(公告)号：US20240031624A1

公开(公告)日：2024-01-25

申请号：US18482654

申请日：2023-10-06

Applicant: MICROSOFT TECHNOLOGY LICENSING, LLC

Inventor： Raymond Kirk PRICE ,  Michael BLEYER ,  Christopher Douglas EDMONDS

IPC: H04N21/2343 ,  G06T3/40 ,  H04N21/242 ,  H04N23/90 ,  H04N23/68

CPC classification number: H04N21/234363 ,  G06T3/4053 ,  H04N21/234381 ,  H04N21/242 ,  H04N23/90 ,  H04N23/689

Abstract: A system for generating high-resolution video from low-resolution images is configured to access a first video stream and a second video stream capturing an environment. The first video stream is captured by a first video capture device. The second video stream is captured by a second video capture device. Image frames of the first video stream are temporally synchronized with corresponding image frames of the second video stream. The system is also configured to generate a composite video stream with a higher resolution than the first or second video streams. Each composite image frame of the composite video stream is generated using a respective image frame of the first video stream and a temporally synchronized corresponding image frame of the second video stream as input.

公开(公告)号：US20230274455A1

公开(公告)日：2023-08-31

申请号：US18295768

申请日：2023-04-04

Applicant: Microsoft Technology Licensing, LLC

Inventor： Raymond Kirk PRICE ,  Michael BLEYER ,  Christopher Douglas EDMONDS

IPC: G06T7/593 ,  G02B27/01 ,  G06T3/40 ,  G06T5/20 ,  G06T5/50

CPC classification number: G06T7/593 ,  G02B27/017 ,  G06T3/40 ,  G06T5/20 ,  G06T5/50 ,  G06T2207/10012 ,  G06T2207/10048 ,  G06T2207/20028

Abstract: A system for low compute high-resolution depth map generation using low-resolution cameras is configured to obtain a stereo pair of images and generate a depth map by performing stereo matching on the stereo pair of images. The system is also configured to obtain a first image comprising first texture information for the environment that has a first image resolution that is higher than an image resolution of images of the stereo pair of images. The system is further configured to generate a reprojected first image by reprojecting the first image to correspond to an image capture perspective associated with the depth map. The reprojection of the first image is based on depth information from the depth map and includes reprojected first texture information for the environment. The system is also configured to generate an upsampled depth map based on the depth map.

公开(公告)号：US20230267577A1

公开(公告)日：2023-08-24

申请号：US18142418

申请日：2023-05-02

Applicant: Microsoft Technology Licensing, LLC

Inventor： Raymond Kirk PRICE ,  Michael BLEYER ,  Christopher Douglas EDMONDS

IPC: G06T5/00 ,  G06T5/50 ,  H04N5/265 ,  G06T3/40 ,  H04N23/951 ,  H04N23/84

CPC classification number: G06T5/003 ,  G06T5/50 ,  H04N5/265 ,  G06T3/4053 ,  H04N23/951 ,  H04N23/843 ,  G06T2207/10024 ,  G06T2207/20201

Abstract: Techniques for using motion data to generate a high resolution output color image from multiple images having sparse color information are disclosed. A camera generates multiple images. The camera's sensor is configured to have a sparse Bayer pattern. While the camera is generating the images, IMU data for each image is acquired. The IMU data indicates a corresponding pose the camera was in while the camera generated each image. The images and the IMU data are fed as input into a motion model. The motion model performs temporal filtering on the images and uses the IMU data to generate a red-only image, a green-only image, and a blue-only image. A high resolution output color image is generated by combining the red-only image, the green-only image, and the blue-only image.

公开(公告)号：US20230254603A1

公开(公告)日：2023-08-10

申请号：US18299201

申请日：2023-04-12

Applicant: Microsoft Technology Licensing, LLC

Inventor： Raymond Kirk PRICE ,  Michael BLEYER ,  Christopher Douglas EDMONDS

IPC: H04N25/705 ,  G02B27/01

CPC classification number: H04N25/705 ,  G02B27/0172 ,  G02B2027/0138

Abstract: A system for structured light depth computation using single photon avalanche diodes (SPADs) is configurable to, over a frame capture time period, selectively activate the illuminator to perform interleaved structured light illumination operations. The interleaved structured light illumination operations comprise alternately emitting at least a first structured light pattern from the illuminator and emitting at least a second structured light pattern from the illuminator. The system is also configurable to, over the frame capture time period, perform a plurality of sequential shutter operations to configure each SPAD pixel of the SPAD array to enable photon detection. The plurality of sequential shutter operations generates, for each SPAD pixel of the SPAD array, a plurality of binary counts indicating whether a photon was detected during each of the plurality of sequential shutter operations.

公开(公告)号：US20230214962A1

公开(公告)日：2023-07-06

申请号：US18108788

申请日：2023-02-13

Applicant: Microsoft Technology Licensing, LLC

Inventor： Raymond Kirk PRICE ,  Michael BLEYER ,  Christopher Douglas EDMONDS

IPC: G06T3/40 ,  H04N23/10 ,  H04N23/951

CPC classification number: G06T3/4061 ,  H04N23/10 ,  H04N23/951 ,  G06T3/4007

Abstract: Techniques for generating a high resolution full color output image from lower resolution sparse color input images are disclosed. A camera generates images. The camera's sensor has a sparse Bayer pattern. While the camera is generating the images, IMU data for each image is acquired. The IMU data indicates a corresponding pose the camera was in while the camera generated each image. The images and IMU data are fed into a motion model, which performs temporal filtering on the images and uses the IMU data to generate a red-only image, a green-only image, a blue-only image, and a monochrome image. The color images are up-sampled to match the resolution of the monochrome image. A high resolution output color image is generated by combining the up-sampled images and the monochrome image.

公开(公告)号：US20230148231A1

公开(公告)日：2023-05-11

申请号：US17524270

申请日：2021-11-11

Applicant: Microsoft Technology Licensing, LLC

Inventor： Raymond Kirk PRICE ,  Michael BLEYER ,  Christopher Douglas EDMONDS ,  Sudipta Narayan SINHA

IPC: G06T15/20 ,  G06T7/33 ,  G06T7/00

CPC classification number: G06T15/205 ,  G06T7/33 ,  G06T7/97

Abstract: Techniques for aligning images generated by two cameras are disclosed. This alignment is performed by computing a relative 3D orientation between the two cameras. A first gravity vector for a first camera and a second gravity vector for a second camera are determined. A first camera image is obtained from the first camera, and a second camera image is obtained from the second camera. A first alignment process is performed to partially align the first camera's orientation with the second camera's orientation. This process is performed by aligning the gravity vectors, thereby resulting in two degrees of freedom of the relative 3D orientation being eliminated. Visual correspondences between the two images are identified. A second alignment process is performed to fully align the orientations. This process is performed by using the identified visual correspondences to identify and eliminate a third degree of freedom of the relative 3D orientation.

公开(公告)号：US20230145672A1

公开(公告)日：2023-05-11

申请号：US17524274

申请日：2021-11-11

Applicant: Microsoft Technology Licensing, LLC

Inventor： Raymond Kirk PRICE ,  Christopher Douglas EDMONDS ,  Michael BLEYER ,  Christian Markus MAEKELAE

IPC: G06T5/50 ,  G06T3/40 ,  G06T5/00 ,  H04N5/232

CPC classification number: G06T5/50 ,  G06T3/40 ,  G06T5/002 ,  H04N5/23258 ,  H04N5/23264 ,  G06T2207/20216 ,  G06T2207/20224

Abstract: Techniques to temporally filter images via a filtering weight computation are disclosed. A first image having a first timestamp and a second image having a second timestamp are acquired. These images are generated by a camera, and the first timestamp is before the second timestamp. A motion compensation (MC) operation is performed on the first image to produce an MC image. A difference image is generated using the MC image and the second image. The difference image reflects differences in intensities that exist between the two images. A local weight map is generated based on those differences. A global weight map is generated based on certain IMU data. A final weight map is generated based on a combination of the local weight map and the global weight map. The final weight map is used to generate a filtered image.

公开(公告)号：US20230122185A1

公开(公告)日：2023-04-20

申请号：US17504123

申请日：2021-10-18

Applicant: Microsoft Technology Licensing, LLC

Inventor： Raymond Kirk PRICE ,  Michael BLEYER ,  Christopher Douglas EDMONDS ,  Carlos Andre TAVARES CAMPOS

IPC: H04N13/156 ,  G06T7/593 ,  G06T7/521 ,  H04N13/332 ,  H04N13/243 ,  H04N13/271 ,  G06T7/30 ,  G02B27/01 ,  G01B11/22 ,  G01C3/02

Abstract: Techniques for performing a hardware-based image alignment process are disclosed. A relative position and orientation between a system camera and a detached external camera are determined. This process is performed using 6 degree of freedom (DOF) trackers on the system camera and on the external camera. A depth measurement, which indicates a distance between the external camera and a scene where the external camera is aimed, is obtained. The system camera generates a system camera image, and the external camera generates an image. An overlaid image is generated by using the relative position and orientation and the depth measurement to reproject the second content onto the first content.