公开(公告)号：US20240070812A1

公开(公告)日：2024-02-29

申请号：US18358857

申请日：2023-07-25

Applicant: QUALCOMM Incorporated

Inventor： Risheek GARREPALLI ,  Rajeswaran CHOCKALINGAPURAMRAVINDRAN ,  Jisoo JEONG ,  Fatih Murat PORIKLI

IPC: G06T3/40 ,  G06T7/579

CPC classification number: G06T3/4053 ,  G06T7/579

Abstract: A processor-implemented method comprises processing a single level cost volume across multiple processing stages by varying a receptive field across each of the processing stages. The method also includes performing a learning-based correspondence estimation task based on the processing. The varying may include processing a different resolution of the cost volume at each processing stage while maintaining a same neighborhood sampling radius. The resolution may increase from a first processing stage to a later processing stage. The varying may also include varying a neighborhood sampling radius at each of the processing stages while maintaining a same resolution. The task may be optical flow estimation or stereo estimation.

公开(公告)号：US20250148633A1

公开(公告)日：2025-05-08

申请号：US18666502

申请日：2024-05-16

Applicant: QUALCOMM Incorporated

Inventor： Rajeev YASARLA ,  Hong CAI ,  Risheek GARREPALLI ,  Yinhao ZHU ,  Jisoo JEONG ,  Yunxiao SHI ,  Manish Kumar SINGH ,  Fatih Murat PORIKLI

IPC: G06T7/593 ,  G06T7/20

Abstract: Systems and techniques are provided for generating depth information. For example, a process can include obtaining a first feature volume including visual features corresponding to each respective frame included in a first set of frames. A first query generator network can generate reconstruction features associated with a reconstructed feature volume corresponding to the first feature volume. Based on the first feature volume, a second query generator network can generate motion features associated with predicted future motion corresponding to the first feature volume. An initial depth prediction can be generated for each respective frame based on cross-attention between features of a depth prediction decoder, the reconstruction features, and the motion features. A refined depth prediction can be generated for each respective based on cross-attention between the initial depth prediction, the reconstruction features, and the motion features.

公开(公告)号：US20240303841A1

公开(公告)日：2024-09-12

申请号：US18538869

申请日：2023-12-13

Applicant: QUALCOMM Incorporated

Inventor： Rajeev YASARLA ,  Hong CAI ,  Jisoo JEONG ,  Risheek GARREPALLI ,  Yunxiao SHI ,  Fatih Murat PORIKLI

IPC: G06T7/50 ,  G06T7/246 ,  G06T11/60 ,  G06V10/44 ,  G06V10/62

CPC classification number: G06T7/50 ,  G06T7/246 ,  G06T11/60 ,  G06V10/44 ,  G06V10/62

Abstract: Disclosed are systems and techniques for capturing images (e.g., using a monocular image sensor) and detecting depth information. According to some aspects, a computing system or device can generate a feature representation of a current image and update accumulated feature information for storage in a memory based on a feature representation of a previous image and optical flow information of the previous image. The accumulated feature information can include accumulated image feature information associated with a plurality of previous images and accumulated optical flow information associated of the plurality of previous images. The computing system or device can obtain information associated with relative motion of the current image based on the accumulated feature information and the feature representation of the current image. The computing system or device can estimate depth information for the current image based on the information associated with the relative motion and the accumulated feature information.

公开(公告)号：US20240177329A1

公开(公告)日：2024-05-30

申请号：US18481050

申请日：2023-10-04

Applicant: QUALCOMM Incorporated

Inventor： Hong CAI ,  Yinhao ZHU ,  Jisoo JEONG ,  Yunxiao SHI ,  Fatih Murat PORIKLI

IPC: G06T7/593 ,  G06T3/40 ,  G06T7/246 ,  G06T7/579

CPC classification number: G06T7/593 ,  G06T3/40 ,  G06T7/248 ,  G06T7/579 ,  G06T2207/10012 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: Systems and techniques are provided for processing sensor data. For example, a process can include determining, using a trained machine learning system, a predicted depth map for an image, the predicted depth map including a respective predicted depth value for each pixel of the image. The process can further include obtaining depth values for the image, the depth values including depth values for less than all pixels of the image from a tracker configured to determine the depth values based on one or more feature points between frames. The process can further include scaling the predicted depth map for the image using and the depth values. The output of the process can be scale-correct depth prediction values.

公开(公告)号：US20220156946A1

公开(公告)日：2022-05-19

申请号：US17510763

申请日：2021-10-26

Applicant: QUALCOMM Incorporated

Inventor： Jamie Menjay LIN ,  Jisoo JEONG ,  Fatih Murat PORIKLI

IPC: G06T7/246 ,  G06T3/60 ,  G06N3/08

Abstract: Systems and techniques are described for performing supervised learning (e.g., semi-supervised learning, self-supervised learning, and/or mixed supervision learning) for optical flow estimation. For example, a method can include obtaining an image associated with a sequence of images and generating an occluded image. The occluded image can include at least one of the image with an occlusion applied to the image and a different image of the sequence of images with the occlusion applied. The method can include determining a matching map based at least on matching areas of the image and the occluded image and, based on the matching map, determining a loss term associated with an optical flow loss prediction associated with the image and the occluded image. The loss term may include a matched loss and/or other loss. Based on the loss term, the method can include training a network configured to determine an optical flow between images.

公开(公告)号：US20250139733A1

公开(公告)日：2025-05-01

申请号：US18499604

申请日：2023-11-01

Applicant: QUALCOMM Incorporated

Inventor： Jisoo JEONG ,  Hong CAI ,  Risheek GARREPALLI ,  Jamie Menjay LIN ,  Munawar HAYAT ,  Fatih Murat PORIKLI

IPC: G06T3/00 ,  G06T5/50 ,  G06T7/20

Abstract: Systems and techniques described herein relate to generating an inter-frame from a first and second frame. An apparatus includes a memory storing a first frame and a second frame; and a processor coupled to the memory and configured to: estimate at least one optical flow between the first frame and the second frame; generate, based on the at least one optical flow, at least one occlusion mask; generate, based on the at least one optical flow and the at least one occlusion mask, at least one weighting mask; generate, based on the at least one optical flow and the at least one weighting mask, at least one inter-frame optical flow; generate, based on the at least one inter-frame optical flow and at least one of the first frame or the second frame, at least one warped frame; and generate, based on the at least one warped frame, an inter-frame.

公开(公告)号：US20250095182A1

公开(公告)日：2025-03-20

申请号：US18468656

申请日：2023-09-15

Applicant: QUALCOMM Incorporated

Inventor： Jisoo JEONG ,  Hong CAI ,  Babak EHTESHAMI BEJNORDI ,  Risheek GARREPALLI ,  Rajeev YASARLA ,  Fatih Murat PORIKLI

IPC: G06T7/593 ,  G06T7/246 ,  G06T7/285

Abstract: Techniques and systems are provided for image processing. For instance, a process can include correlating a first set of features from a first viewpoint with a second set of features from a second viewpoint at a first time period to generate a first disparity cost volume; correlating a third set of features from the first viewpoint at a second time period with the first set of features to generate a first optical flow cost volume; gating the first disparity cost volume to generate first intermediate disparity information; gating the first optical flow cost volume to generate first intermediate optical flow information; correlating the first set of features, the second set of features, and the first intermediate optical flow information to generate disparity information for output; and correlating the third set of features, the first set of features, and the first intermediate disparity information to generate optical flow information for output.

公开(公告)号：US20250131606A1

公开(公告)日：2025-04-24

申请号：US18492572

申请日：2023-10-23

Applicant: QUALCOMM Incorporated

Inventor： Shubhankar Mangesh BORSE ,  Risheek GARREPALLI ,  Qiqi HOU ,  Jisoo JEONG ,  Shreya KADAMBI ,  Munawar HAYAT ,  Fatih Murat PORIKLI

IPC: G06T11/00 ,  G06T3/40

Abstract: A processor-implemented method includes receiving a text-semantic input at a first stage of a neural network, including a first convolutional block and no attention layers. The method receives, at a second stage, a first output from the first stage. The second stage comprises a first down sampling block including a first attention layer and a second convolutional block. The method receives, at a third stage, a second output from the second stage. The third stage comprises a first up sampling block including a second attention layer and a first set of convolutional blocks. The method receives, at a fourth stage, the first output from the first stage and a third output from the third stage. The fourth stage comprises a second up sampling block including no attention layers and a second set of convolutional blocks. The method generates an image at the fourth stage, based on the text-semantic input.

公开(公告)号：US20250131325A1

公开(公告)日：2025-04-24

申请号：US18492492

申请日：2023-10-23

Applicant: QUALCOMM Incorporated

Inventor： Risheek GARREPALLI ,  Shubhankar Mangesh BORSE ,  Jisoo JEONG ,  Qiqi HOU ,  Shreya KADAMBI ,  Munawar HAYAT ,  Fatih Murat PORIKLI

IPC: G06N20/00

Abstract: A method for training a diffusion model includes compressing the diffusion model by removing at least one of: one or more model parameters or one or more giga multiply-accumulate operations (GMACs). The method also includes performing guidance conditioning to train the compressed diffusion model, the guidance conditioning combining a conditional output and an unconditional output from respective teacher models. The method further includes performing, after the guidance conditioning, step distillation on the compressed diffusion model.

公开(公告)号：US20250131277A1

公开(公告)日：2025-04-24

申请号：US18492529

申请日：2023-10-23

Applicant: QUALCOMM Incorporated

Inventor： Risheek GARREPALLI ,  Shubhankar Mangesh BORSE ,  Jisoo JEONG ,  Qiqi HOU ,  Shreya KADAMBI ,  Munawar HAYAT ,  Fatih Murat PORIKLI

IPC: G06N3/09

Abstract: A method for training a control neural network includes initializing a baseline diffusion model for training the control neural network, each stage of a control neural network training pipeline corresponding to an element of the baseline diffusion model. The method also includes training, the control neural network, in a stage-wise manner, each stage of the control neural network training pipeline receiving an input from a previous stage of the control neural network training pipeline and the corresponding element of the diffusion model.