公开(公告)号：US20220335718A1

公开(公告)日：2022-10-20

申请号：US17716996

申请日：2022-04-08

Applicant: The Toronto-Dominion Bank

Inventor： Junwei Ma ,  Satya Krishna Gorti ,  Maksims Volkovs ,  Guangwei Yu

IPC: G06V20/40 ,  G06V10/764 ,  G06V10/774 ,  G06V20/70 ,  G06V10/82

Abstract: A video localization system localizes actions in videos based on a classification model and an actionness model. The classification model is trained to make predictions of which segments of a video depict an action and to classify the actions in the segments. The actionness model predicts whether any action is occurring in each segment, rather than predicting a particular type of action. This reduces the likelihood that the video localization system over-relies on contextual information in localizing actions in video. Furthermore, the classification model and the actionness model are trained based on weakly-labeled data, thereby reducing the cost and time required to generate training data for the video localization system.

公开(公告)号：US20250131718A1

公开(公告)日：2025-04-24

申请号：US18988381

申请日：2024-12-19

Applicant: THE TORONTO-DOMINION BANK

Inventor： Junwei Ma ,  Satya Krishna Gorti ,  Maksims Volkovs ,  Guangwei Yu

IPC: G06V20/40 ,  G06V10/764 ,  G06V10/774 ,  G06V10/82 ,  G06V20/70

Abstract: A video localization system localizes actions in videos based on a classification model and an actionness model. The classification model is trained to make predictions of which segments of a video depict an action and to classify the actions in the segments. The actionness model predicts whether any action is occurring in each segment, rather than predicting a particular type of action. This reduces the likelihood that the video localization system over-relies on contextual information in localizing actions in video. Furthermore, the classification model and the actionness model are trained based on weakly-labeled data, thereby reducing the cost and time required to generate training data for the video localization system.

公开(公告)号：US20230131935A1

公开(公告)日：2023-04-27

申请号：US17969505

申请日：2022-10-19

Applicant: THE TORONTO-DOMINION BANK

Inventor： Maksims Volkovs ,  Cheng Chang ,  Guangwei Yu ,  Himanshu Rai ,  Yichao Lu

IPC: G06V10/764 ,  G06V10/77 ,  G06V10/774

Abstract: An object detection model and relationship prediction model are jointly trained with parameters that may be updated through a joint backbone. The offset detection model predicts object locations based on keypoint detection, such as a heatmap local peak, enabling disambiguation of objects. The relationship prediction model may predict a relationship between detected objects and be trained with a joint loss with the object detection model. The loss may include terms for object connectedness and model confidence, enabling training to focus first on highly-connected objects and later on lower-confidence items.

公开(公告)号：US11475059B2

公开(公告)日：2022-10-18

申请号：US16917422

申请日：2020-06-30

Applicant: THE TORONTO-DOMINION BANK

Inventor： Chundi Liu ,  Guangwei Yu ,  Maksims Volkovs

IPC: G06F16/58 ,  G06N3/08 ,  G06N3/04

Abstract: A content retrieval system uses a graph neural network architecture to determine images relevant to an image designated in a query. The graph neural network learns a new descriptor space that can be used to map images in the repository to image descriptors and the query image to a query descriptor. The image descriptors characterize the images in the repository as vectors in the descriptor space, and the query descriptor characterizes the query image as a vector in the descriptor space. The content retrieval system obtains the query result by identifying a set of relevant images associated with image descriptors having above a similarity threshold with the query descriptor.

公开(公告)号：US20220318298A1

公开(公告)日：2022-10-06

申请号：US17848122

申请日：2022-06-23

Applicant: THE TORONTO-DOMINION BANK

Inventor： Maksims Volkovs ,  Cheng Chang ,  Guangwei Yu ,  Chundi Liu

IPC: G06F16/583 ,  G06F16/903 ,  G06F16/58 ,  G06F16/901 ,  G06F16/9032

Abstract: An image retrieval system receives an image for which to identify relevant images from an image repository. Relevant images may be of the same environment or object and features and other characteristics. Images in the repository are represented in an image retrieval graph by a set of image nodes connected by edges to other related image nodes with edge weights representing the similarity of the nodes to each other. Based on the received image, the image traversal system identifies an image in the image retrieval graph and alternatively explores and traverses (also termed “exploits”) the image nodes with the edge weights. In the exploration step, image nodes in an exploration set are evaluated to identify connected nodes that are added to a traversal set of image nodes. In the traversal step, the relevant nodes in the traversal set are added to the exploration set and a query result set.

公开(公告)号：US20210049202A1

公开(公告)日：2021-02-18

申请号：US16917422

申请日：2020-06-30

Applicant: THE TORONTO-DOMINION BANK

Inventor： Chundi Liu ,  Guangwei Yu ,  Maksims Volkovs

IPC: G06F16/58 ,  G06N3/04 ,  G06N3/08

Abstract: A content retrieval system uses a graph neural network architecture to determine images relevant to an image designated in a query. The graph neural network learns a new descriptor space that can be used to map images in the repository to image descriptors and the query image to a query descriptor. The image descriptors characterize the images in the repository as vectors in the descriptor space, and the query descriptor characterizes the query image as a vector in the descriptor space. The content retrieval system obtains the query result by identifying a set of relevant images associated with image descriptors having above a similarity threshold with the query descriptor.