公开(公告)号：WO2019143635A1

公开(公告)日：2019-07-25

申请号：PCT/US2019/013709

申请日：2019-01-15

Applicant: VERILY LIFE SCIENCES LLC

Inventor： BARRAL, Joëlle K. ,  HABBECKE, Martin ,  JIN, Xing ,  YANG, Lin

IPC: A61B34/00 ,  A61B34/30 ,  A61B34/37

CPC classification number: A61B34/30 ,  A61B1/3132 ,  A61B34/25 ,  A61B34/35 ,  A61B34/37 ,  A61B2017/00128 ,  A61B2017/00203 ,  A61B2034/2057 ,  A61B2034/252 ,  A61B2034/254 ,  A61B2034/256 ,  A61B2034/301 ,  A61B2034/302 ,  A61B2090/374 ,  A61B2090/3762 ,  G06N3/08 ,  G06N20/00 ,  G16H30/20

Abstract: A system for robot-assisted surgery includes an image sensor and a display. The system further includes a controller coupled to the image sensor and the display, where the controller includes logic that when executed by the controller causes the system to perform operations. The operations may include acquiring first images of a surgical procedure with the image sensor, and analyzing the first images with the controller to identify a surgical step in the surgical procedure. The operations may further include displaying second images on the display in response to identifying the surgical step; the second images may include at least one of a diagram of human anatomy, a preoperative image, an intraoperative image, or an annotated image of one of the first images.

公开(公告)号：WO2021097241A1

公开(公告)日：2021-05-20

申请号：PCT/US2020/060456

申请日：2020-11-13

Applicant: VERILY LIFE SCIENCES LLC

Inventor： JIN, Xing ,  DONHOWE, Caitlin ,  HABBECKE, Martin ,  BARRAL, Joelle

IPC: G06T7/73 ,  A61B90/00 ,  G16H20/40

Abstract: Systems and methods for depth detection and virtual modeling using surgical robots during a surgical procedure are described. The robotic surgical systems include robotic arms with interchangeable surgical tools. An endoscope at the end of one of the robotic arms includes a depth sensor for detecting a distance from the camera to patient anatomy. The depth data and image data are used to generate a feature model of the patient anatomy for reference by a surgeon. The feature model is displayed to the surgeon in an augmented reality view with preoperative and intraoperative images mapped thereon for reference and guidance of a surgical procedure.

公开(公告)号：WO2020146501A1

公开(公告)日：2020-07-16

申请号：PCT/US2020/012740

申请日：2020-01-08

Applicant: VERILY LIFE SCIENCES LLC

Inventor： JIN, Xing ,  BARRAL, Joëlle

IPC: A61B5/06 ,  G06K9/00 ,  G06Q50/10

Abstract: One example method for detecting phases of a surgical procedure via video processing includes accessing a video of the surgical procedure and dividing the video into one or more blocks, each of the blocks containing one or more video frames. For each of the blocks, the method includes applying a prediction model on the video frames of the respective block to obtain a phase prediction for each of the video frames. The prediction model is configured to predict, for an input video frame, one of the plurality of phases of the surgical procedure. The method further includes generating an aggregated phase prediction for the respective block by aggregating the phase predictions of the video frames, and modifying the video of the surgical procedure to include an indication of a predicted phase of the respective block based on the aggregated phase prediction.

公开(公告)号：WO2019226440A1

公开(公告)日：2019-11-28

申请号：PCT/US2019/032530

申请日：2019-05-15

Applicant: VERILY LIFE SCIENCES LLC

Inventor： BARRAL, Joëlle K. ,  CAMPION, Gianni ,  HABBECKE, Martin ,  JIN, Xing

IPC: G09B23/28

Abstract: A simulator for simulating soft body deformation includes a display system, a user interface, and a controller. The controller includes one or more processors coupled to memory that stores instructions that when executed cause the system to perform operations. The operations include generating a simulated soft body having a shape represented, at least in part, by a plurality of nodes. The operations further include determining, with at least a first machine learning model, a displacement of individual nodes included in the plurality of nodes in response to a simulated force applied to the simulated soft body. The operations further include rendering a deformation of the simulated soft body in substantially real time in response to the simulated force based, at least in part, on the displacement.

公开(公告)号：WO2019204777A1

公开(公告)日：2019-10-24

申请号：PCT/US2019/028389

申请日：2019-04-19

Applicant: VERILY LIFE SCIENCES LLC

Inventor： JIN, Xing ,  BARRAL, Joëlle K. ,  YANG, Lin ,  HABBECKE, Martin ,  CAMPION, Gianni

IPC: G09B9/00 ,  G09B23/28 ,  G06K9/00

Abstract: A surgical simulator for simulating a surgical scenario comprises a display system, a user interface, and a controller. The controller includes one or more processors coupled to memory that stores instructions that when executed cause the system to perform operations. The operations include generating simulated surgical videos, each representative of the surgical scenario. The operations further include associating simulated ground truth data from the simulation with the simulated surgical videos. The ground truth data corresponds to context information of at least one of a simulated surgical instrument, a simulated anatomical region, a simulated surgical task, or a simulated action. The operations further include annotating features of the simulated surgical videos based, at least in part, on the simulated ground truth data for training a machine learning model.

公开(公告)号：WO2020197799A1

公开(公告)日：2020-10-01

申请号：PCT/US2020/022540

申请日：2020-03-13

Applicant: VERILY LIFE SCIENCES LLC

Inventor： BARRAL, Joëlle ,  HABBECKE, Martin ,  YANG, Lin ,  JIN, Xing

IPC: H04N21/2343 ,  H04N21/254 ,  H04N21/258 ,  H04N21/278 ,  G06Q10/06

Abstract: One example method for identifying useful segments in surgical videos includes accessing a video of a surgical procedure and user activities of a plurality of users who have watched the video of the surgical procedure. The user activities include operations performed during playback of the video. The method further includes dividing the video into multiple segments and determining a popularity score for each of the multiple segments based on the operations. Useful segments are identified from the segments based on the popularity scores. The method further includes generating metadata for the video of the surgical procedure to include an indication of the identified useful segments and associating the metadata with the video of the surgical procedure.

公开(公告)号：WO2020117561A2

公开(公告)日：2020-06-11

申请号：PCT/US2019/063300

申请日：2019-11-26

Applicant: VERILY LIFE SCIENCES LLC

Inventor： DONHOWE, Caitlin ,  BARRAL, Joëlle ,  JIN, Xing

IPC: A61B34/30 ,  A61B34/37

Abstract: One example method for improving robotic surgical safety via video processing includes identifying, during a robotic surgical procedure, one or more surgical tools using one or more images of the surgical procedure captured by a camera, the robotic surgical procedure employing a robotic surgical device controlling the one or more identified surgical tools; predicting, for at least one of the one or more images, one or more loaded surgical tools that are controlled by the robotic surgical device and should be in a field of view of the camera; comparing the one or more identified surgical tools with the one or more loaded surgical tools; determining that at least one of the one or more loaded surgical tools does not match any of the one or more identified surgical tools; and causing the at least one loaded tool of the robotic surgical device to be disabled.

公开(公告)号：WO2019182875A1

公开(公告)日：2019-09-26

申请号：PCT/US2019/022352

申请日：2019-03-14

Applicant: VERILY LIFE SCIENCES LLC

Inventor： HABBECKE, Martin ,  BARRAL, Joëlle K. ,  YANG, Lin ,  JIN, Xing

IPC: G16H30/20 ,  G16H20/40

Abstract: A system includes a display, and a database including surgical videos, images of organs in a human body obtained from a medical imaging device, and images of disease in a human body obtained from the medical imaging device. A controller including a processor is coupled to memory, the database, and the display, and the memory stores information that when executed by the processor causes the system to perform operations. For example, the processor may determine first organ information from the images of the organs, and first disease information from the images of the disease. The processor my calculate a similarity score between the first organ information and the first disease information and second disease information and second organ information indexed to the surgical videos. The processor selects one or more of the surgical videos based on the similarity score, and displays the surgical videos on the display.