公开(公告)号：US20240428428A1

公开(公告)日：2024-12-26

申请号：US18593067

申请日：2024-03-01

Applicant: INTUITIVE SURGICAL OPERATIONS, INC.

Inventor： Rui Guo ,  Xi Liu ,  Ziheng Wang ,  Anthony M. Jarc

IPC: G06T7/246 ,  A61B34/20 ,  G06T7/73

Abstract: A method for predicting movement of a first plurality of keypoints of a first instrument comprises receiving, at a neural network model, a first location of the first plurality of keypoints and a first location of a second plurality of keypoints of a second instrument. The method further comprises determining a trajectory for the first and second pluralities of keypoints by: generating, using an attention model of the neural network model, a first and second tool-level graph indicating a spatial-temporal relationship between the first and second pluralities of keypoints, respectively; and generating a scene-level graph based on the tool-level graphs. The scene-level graph indicates a spatial-temporal relationship between the first and second pluralities of keypoints. The method further comprises generating an output image based on the determined trajectory. The output image includes an output location of the first and second pluralities of keypoints.

公开(公告)号：US20230368530A1

公开(公告)日：2023-11-16

申请号：US18035089

申请日：2021-11-17

Applicant: Intuitive Surgical Operations, Inc.

Inventor： Ziheng Wang ,  Kiran Bhattacharyya ,  Anthony Jarc

IPC: G06V20/40 ,  G06V10/764 ,  G06V10/776 ,  G06V10/82 ,  G06V10/80

CPC classification number: G06V20/41 ,  G06V10/764 ,  G06V10/776 ,  G06V10/82 ,  G06V10/811 ,  G06V2201/03 ,  G16H40/20

Abstract: Various of the disclosed embodiments relate to systems and methods for recognizing types of surgical operations from data gathered in a surgical theater, such as recognizing a surgery procedure and corresponding specialty from endoscopic video data. Some embodiments select discrete frame sets from the data for individual consideration by a corpus of machine learning models, Some embodiments may include an uncertainty indication with each classification to guide downstream decision-making based upon the classification. For example, where the system is used as part of a data annotation pipeline, uncertain classifications may be flagged for downstream confirmation and review by a human reviewer.

公开(公告)号：US20240296604A1

公开(公告)日：2024-09-05

申请号：US18592922

申请日：2024-03-01

Applicant: INTUITIVE SURGICAL OPERATIONS, INC.

Inventor： Rui Guo ,  Xi Liu ,  Ziheng Wang ,  Marzieh Ershad Langroodi ,  Anthony M. Jarc

IPC: G06T11/20 ,  A61B34/20 ,  G06T5/60 ,  G06T5/70 ,  G06T7/00 ,  G06T7/246 ,  G06T7/73

CPC classification number: G06T11/206 ,  A61B34/20 ,  G06T5/60 ,  G06T5/70 ,  G06T7/0012 ,  G06T7/246 ,  G06T7/73 ,  A61B2034/2065 ,  G06T2207/10016 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/30004 ,  G06T2207/30241 ,  G06T2210/41

Abstract: A method for detecting a location of a plurality of keypoints of a surgical instrument comprises receiving, at a first neural network model, a video input of a surgical procedure. The method further comprises generating, using the first neural network model, a first output image including a first output location of the plurality of keypoints annotated on a first output image of the surgical instrument. The method further comprises receiving, at a second neural network model, the first output image and historic keypoint trajectory data including a historic trajectory for the plurality of keypoints. The method further comprises determining, using the second neural network model, a trajectory for the plurality of keypoints. The method further comprises generating, using the second neural network model, a second output image including a second output location of the plurality of keypoints annotated on a second output image of the surgical instrument.

公开(公告)号：US20230316756A1

公开(公告)日：2023-10-05

申请号：US18035078

申请日：2021-11-18

Applicant: Intuitive Surgical Operations, Inc.

Inventor： Ziheng Wang ,  Kiran Bhattacharyya ,  Samuel Bretz ,  Anthony Jarc ,  Xi Liu ,  Andrea Villa ,  Aneeq Zia

IPC: G06V20/50 ,  G06V20/40 ,  G06V10/82 ,  G06V10/80 ,  G06V10/77

CPC classification number: G06V20/50 ,  G06V20/41 ,  G06V20/46 ,  G06V10/82 ,  G06V10/809 ,  G06V10/7715 ,  G06V2201/034 ,  G16H30/40

Abstract: Various of the disclosed embodiments relate to systems and methods for processing surgical data to facilitate further downstream operations. For example, some embodiments may include machine learning systems trained to recognize whether video from surgical visualization tools, such as endoscopes, depicts a field of view inside or outside the patient body. The system may excise or whiteout frames of video appearing outside the patient so as to remove potentially compromising personal information, such as the identities of members of the surgical team, the patients identity, configurations of the surgical theater, etc. Appropriate removal of such non-surgical data may facilitate downstream processing, e.g., by complying with regulatory requirements as well as by removing extraneous data potentially inimical to further downstream processing, such as training a downstream classifier.