公开(公告)号：US20240119555A1

公开(公告)日：2024-04-11

申请号：US18527528

申请日：2023-12-04

申请人： Google LLC

发明人： Junjie Ke ,  Feng Yang ,  Qifei Wang ,  Yilin Wang ,  Peyman Milanfar

IPC分类号： G06T3/00 ,  G06T3/40 ,  G06T7/00

CPC分类号： G06T3/0012 ,  G06T3/40 ,  G06T7/0002 ,  G06T2207/20016 ,  G06T2207/20081 ,  G06T2207/30168

摘要： The technology employs a patch-based multi-scale Transformer (300) that is usable with various imaging applications. This avoids constraints on image fixed input size and predicts the quality effectively on a native resolution image. A native resolution image (304) is transformed into a multi-scale representation (302), enabling the Transformer's self-attention mechanism to capture information on both fine-grained detailed patches and coarse-grained global patches. Spatial embedding (316) is employed to map patch positions to a fixed grid, in which patch locations at each scale are hashed to the same grid. A separate scale embedding (318) is employed to distinguish patches coming from different scales in the multiscale representation. Self-attention (508) is performed to create a final image representation. In some instances, prior to performing self-attention, the system may prepend a learnable classification token (322) to the set of input tokens.

公开(公告)号：US11954834B2

公开(公告)日：2024-04-09

申请号：US17815445

申请日：2022-07-27

申请人： BOSTON SCIENTIFIC SCIMED, INC.

发明人： Niraj Prasad Rauniyar ,  Robert J. Riker ,  Timothy Paul Harrah

IPC分类号： G06T7/246 ,  A61B1/00 ,  A61B5/06 ,  G06T3/00 ,  G06T3/4053 ,  G06T5/50 ,  G06T7/32 ,  G06T7/55 ,  G06T7/70 ,  H04N23/50 ,  G06K9/32

CPC分类号： G06T5/50 ,  A61B1/00045 ,  A61B5/065 ,  G06T3/0012 ,  G06T3/0068 ,  G06T3/4053 ,  G06T7/32 ,  G06T7/55 ,  G06T7/70 ,  H04N23/50 ,  G06T2207/10016 ,  G06T2207/10068 ,  G06T2207/20221 ,  G06T2207/30084 ,  H04N23/555

摘要： An endoscopic system includes an endoscopic imager configured to capture image frames of a target site within a living body and a processor configured to apply a spatial transform to a preliminary set of image frames, the spatial transform converting the image frames into cylindrical coordinates; calculate a map image from the spatially transformed image frames, each pixel position in the map image being defined with a vector of fixed dimension; align a current image frame with the map image and apply the spatial transform to the current image frame; fuse the spatially transformed current image frame to the map image to generate a fused image; and apply an inverse spatial transform to the fused image to generate an enhanced current image frame having a greater spatial resolution than the current image frame. The system also includes a display displaying the enhanced current image frame.

公开(公告)号：US20230377093A1

公开(公告)日：2023-11-23

申请号：US18320496

申请日：2023-05-19

申请人： DISNEY ENTERPRISES, INC. ,  ETH Zürich (Eidgenössische Technische Hochschule Zürich)

发明人： Abdelaziz DJELOUAH ,  Michael Yves BERNASCONI ,  Farnood SALEHI ,  Christopher Richard SCHROERS

IPC分类号： G06T3/00 ,  G06T3/40

CPC分类号： G06T3/0093 ,  G06T3/0012 ,  G06T3/4046

摘要： Techniques are disclosed for resampling images. In some embodiments, a resampling model includes (1) one or more feature extraction layers that extract features from an input image and a degradation map; (2) one or more resampling layers that generate warped features from the extracted features and a warp grid; and (3) one or more prediction layers that generate, from the warped features, an output image or resampling kernels that can be applied to the input image to generate an output image. In some embodiments, the resampling model can be trained by applying degradation maps to output images in a training data set to generate corresponding input images, and training the resampling model using the input images and the corresponding output images.

公开(公告)号：US20230351624A1

公开(公告)日：2023-11-02

申请号：US18331174

申请日：2023-06-08

申请人： HUAWEI TECHNOLOGIES CO., LTD.

发明人： Patrick RUHKAMP ,  Onay Urfalioglu

IPC分类号： G06T3/00 ,  G06T7/70 ,  G06T3/40 ,  G06T7/55 ,  G06T7/11

CPC分类号： G06T7/55 ,  G06T3/0012 ,  G06T3/4007 ,  G06T7/11 ,  G06T7/70 ,  G06T2207/10016 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/30244

摘要： A computer implemented method of estimating depth for an image and relative camera poses between images in a video sequence, includes backwards warping the source image to generate a first reconstructed target image, and calculating an initial image reconstruction loss based on the target image and the first reconstructed target image. Forward warping the source depth map is performed to generate a second reconstructed target depth map, and an occlusion mask is generated based on the second reconstructed target depth map. The method further includes regularising the initial image reconstruction loss based on the generated occlusion mask. Thus, an occlusion aware method of image reconstruction is provided via a combination of forward and backward warping which identifies and masks occluded areas, and regularizes the image reconstruction loss.

公开(公告)号：US20230316560A1

公开(公告)日：2023-10-05

申请号：US18190154

申请日：2023-03-27

申请人： HTC Corporation

发明人： Yen-Ting LIU ,  Meng-Ju WU

IPC分类号： G06T7/70 ,  G06T3/00 ,  G06T7/20 ,  G06T7/13 ,  H04N5/262 ,  G06F3/01

CPC分类号： G06T7/70 ,  G06T3/0012 ,  G06T7/20 ,  G06T7/13 ,  H04N5/2628 ,  G06F3/011 ,  G06T2207/30196 ,  G06T2207/10028

摘要： A tracking apparatus, method, and non-transitory computer readable storage medium thereof are provided. The tracking apparatus generates a map information of simultaneous localization and mapping corresponding to a regional space based on a real-time image. The tracking apparatus calculates a first spatial position and a first orientation of a first display related to the image capturing device in the regional space based on the map information. The tracking apparatus calculates a human pose of a first operating user in the regional space. The tracking apparatus transforms the real-time image to generate a first transformed image corresponding to the first operating user based on the first spatial position, the first orientation, and the human pose, wherein the first transformed image is displayed on the first display.

公开(公告)号：US20230260239A1

公开(公告)日：2023-08-17

申请号：US18168355

申请日：2023-02-13

申请人： Meta Platforms, Inc.

发明人： Vincent Charles CHEUNG ,  Jiemin ZHANG ,  Salvatore CANDIDO ,  Hung-Yu TSENG

IPC分类号： G06T19/20 ,  G06V20/20 ,  G06T7/70 ,  G06T3/00

CPC分类号： G06T19/20 ,  G06V20/20 ,  G06T7/70 ,  G06T3/0012 ,  G06T2200/24 ,  G06T2219/2021

摘要： Aspects of the present disclosure are directed to creating a skybox for an artificial reality (“XR”) world from a two-dimensional (“2D”) image. The 2D image is scanned and split into at least two portions. The portions are mapped onto the interior of a virtual enclosed 3D shape, for example, a virtual cube. A generative adversarial network (GAN) interpolates from the information in the areas mapped from the portions to fill in at least some unmapped areas of the interior of the 3D shape. The 3D shape can be placed in a user's XR world to become the skybox surrounding that world.

公开(公告)号：US11727589B2

公开(公告)日：2023-08-15

申请号：US17377684

申请日：2021-07-16

申请人： TOYOTA RESEARCH INSTITUTE, INC.

发明人： Vitor Guizilini ,  Rares Andrei Ambrus ,  Adrien David Gaidon ,  Igor Vasiljevic ,  Gregory Shakhnarovich

IPC分类号： G06T7/55 ,  B60R1/00 ,  G06T3/00 ,  G05D1/02 ,  G06N3/08 ,  G06T7/579 ,  G06T7/292 ,  G06T7/11 ,  B60W60/00 ,  G06T3/40 ,  G06F18/214 ,  H04N23/90

CPC分类号： G06T7/55 ,  B60R1/00 ,  B60W60/001 ,  G05D1/0212 ,  G05D1/0246 ,  G06F18/214 ,  G06F18/2148 ,  G06N3/08 ,  G06T3/0012 ,  G06T3/0093 ,  G06T3/40 ,  G06T7/11 ,  G06T7/292 ,  G06T7/579 ,  H04N23/90 ,  B60R2300/102 ,  B60W2420/42 ,  G05D2201/0213 ,  G06T2207/10028 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/30244 ,  G06T2207/30252

摘要： A method for multi-camera monocular depth estimation using pose averaging is described. The method includes determining a multi-camera photometric loss associated with a multi-camera rig of an ego vehicle. The method also includes determining a multi-camera pose consistency constraint (PCC) loss associated with the multi-camera rig of the ego vehicle. The method further includes adjusting the multi-camera photometric loss according to the multi-camera PCC loss to form a multi-camera PCC photometric loss. The method also includes training a multi-camera depth estimation model and an ego-motion estimation model according to the multi-camera PCC photometric loss. The method further includes predicting a 360° point cloud of a scene surrounding the ego vehicle according to the trained multi-camera depth estimation model and the ego-motion estimation model.

公开(公告)号：US11688090B2

公开(公告)日：2023-06-27

申请号：US17377161

申请日：2021-07-15

申请人： TOYOTA RESEARCH INSTITUTE, INC.

发明人： Vitor Guizilini ,  Rares Andrei Ambrus ,  Adrien David Gaidon ,  Igor Vasiljevic ,  Gregory Shakhnarovich

IPC分类号： G06T7/55 ,  G06N3/08 ,  G06T7/579 ,  B60R1/00 ,  G06T3/00 ,  G05D1/02 ,  G06T7/292 ,  G06T7/11 ,  B60W60/00 ,  G06T3/40 ,  G06F18/214 ,  H04N23/90

CPC分类号： G06T7/55 ,  B60R1/00 ,  B60W60/001 ,  G05D1/0212 ,  G05D1/0246 ,  G06F18/214 ,  G06F18/2148 ,  G06N3/08 ,  G06T3/0012 ,  G06T3/0093 ,  G06T3/40 ,  G06T7/11 ,  G06T7/292 ,  G06T7/579 ,  H04N23/90 ,  B60R2300/102 ,  B60W2420/42 ,  G05D2201/0213 ,  G06T2207/10028 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/30244 ,  G06T2207/30252

摘要： A method for multi-camera self-supervised depth evaluation is described. The method includes training a self-supervised depth estimation model and an ego-motion estimation model according to a multi-camera photometric loss associated with a multi-camera rig of an ego vehicle. The method also includes generating a single-scale correction factor according to a depth map of each camera of the multi-camera rig during a time-step. The method further includes predicting a 360° point cloud of a scene surrounding the ego vehicle according to the self-supervised depth estimation model and the ego-motion estimation model. The method also includes scaling the 360° point cloud according to the single-scale correction factor to form an aligned 360° point cloud.

公开(公告)号：US11663762B2

公开(公告)日：2023-05-30

申请号：US17083899

申请日：2020-10-29

申请人： Adobe Inc.

发明人： Jianming Zhang ,  Zhe Lin ,  Radomir Mech ,  Xiaohui Shen

IPC分类号： G06T11/60 ,  G06T3/00 ,  G06T7/12

CPC分类号： G06T11/60 ,  G06T3/0012 ,  G06T7/12 ,  G06T2207/20132 ,  G06T2210/22

摘要： Embodiments of the present invention are directed to facilitating region of interest preservation. In accordance with some embodiments of the present invention, a region of interest preservation score using adaptive margins is determined. The region of interest preservation score indicates an extent to which at least one region of interest is preserved in a candidate image crop associated with an image. A region of interest positioning score is determined that indicates an extent to which a position of the at least one region of interest is preserved in the candidate image crop associated with the image. The region of interest preservation score and/or the preserving score are used to select a set of one or more candidate image crops as image crop suggestions.

公开(公告)号：US11663704B2

公开(公告)日：2023-05-30

申请号：US17243405

申请日：2021-04-28

申请人： Microsoft Technology Licensing, LLC

发明人： Karlton David Powell

IPC分类号： H04N7/15 ,  G06T5/00 ,  G06T3/00 ,  H04N5/262

CPC分类号： G06T5/006 ,  G06T3/0012 ,  H04N5/2628 ,  H04N7/15

摘要： Examples are disclosed relating to applying an analytical geometric projection that has been modified by an amplitude function. One example provides a computing device comprising a logic subsystem and a storage subsystem holding instructions executable by the logic subsystem to receive an image of a scene as acquired by an image sensor, apply a mapping to the image of the scene that maps pixels of the image to projected pixels on an analytical projection that is modified by an amplitude function such that the analytical projection achieves a higher zoom effect on pixels closer to a center of the image compared to pixels closer to an edge of the image, thereby obtaining a corrected image, and output the corrected image.