公开(公告)号：US20240296604A1

公开(公告)日：2024-09-05

申请号：US18592922

申请日：2024-03-01

Applicant: INTUITIVE SURGICAL OPERATIONS, INC.

Inventor： Rui Guo ,  Xi Liu ,  Ziheng Wang ,  Marzieh Ershad Langroodi ,  Anthony M. Jarc

IPC: G06T11/20 ,  A61B34/20 ,  G06T5/60 ,  G06T5/70 ,  G06T7/00 ,  G06T7/246 ,  G06T7/73

CPC classification number: G06T11/206 ,  A61B34/20 ,  G06T5/60 ,  G06T5/70 ,  G06T7/0012 ,  G06T7/246 ,  G06T7/73 ,  A61B2034/2065 ,  G06T2207/10016 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/30004 ,  G06T2207/30241 ,  G06T2210/41

Abstract: A method for detecting a location of a plurality of keypoints of a surgical instrument comprises receiving, at a first neural network model, a video input of a surgical procedure. The method further comprises generating, using the first neural network model, a first output image including a first output location of the plurality of keypoints annotated on a first output image of the surgical instrument. The method further comprises receiving, at a second neural network model, the first output image and historic keypoint trajectory data including a historic trajectory for the plurality of keypoints. The method further comprises determining, using the second neural network model, a trajectory for the plurality of keypoints. The method further comprises generating, using the second neural network model, a second output image including a second output location of the plurality of keypoints annotated on a second output image of the surgical instrument.

公开(公告)号：US20230316545A1

公开(公告)日：2023-10-05

申请号：US18035100

申请日：2021-11-19

Applicant: Intuitive Surgical Operations, Inc.

Inventor： Xi Liu ,  Marzieh Ershad Langroodi ,  Anthony Jarc

IPC: G06T7/246 ,  G06V10/75 ,  G06V20/40

CPC classification number: G06T7/248 ,  G06V10/751 ,  G06V20/48 ,  G06V20/41 ,  G06T2207/30004 ,  G06T2207/10016 ,  G06V2201/034 ,  G06T2200/24 ,  G06T2207/20081 ,  G06T2207/20084 ,  G16H30/40

Abstract: Various of the disclosed embodiments are directed to systems and computer-implemented methods for determining surgical system events and/or kinematic data based upon surgical video data, such as video acquired at an endoscope. In some embodiments, derived data may be inferred from elements appearing in a graphical user interface (GUI) exclusively. Icons and text may be recognized in the GUI to infer event occurrences and tool actions. In some embodiments, derived data may additionally, or alternatively, be inferred from optical flow values derived from the video and by tracking tools entering and leaving the video field of view. Some embodiments include logic for reconciling data values derived from each of these approaches.