Real-time depth estimation from monocular endoscope images

    公开(公告)号:US09750399B2

    公开(公告)日:2017-09-05

    申请号:US13266465

    申请日:2010-03-25

    摘要: An endoscopic surgical method involves an advancement of an endoscope (20) to a target location within an anatomical region of a body and a generation of a plurality of endoscopic video frames (22) as the endoscope (20) is advanced to the target location with the endoscopic video frames (2) illustrating monocular endoscopic images of the anatomical region. For real-time estimation of a depth of an object within monocular endoscopic images (e.g., depth of a bronchial wall within an monocular endoscopic images of a bronchial tube), the method further involves (S41) a determination of an optical flow of one or more image points within a frame time series of the monocular endoscopic images of the anatomical region, and (S42) an estimation of a depth field indicative of a depth of the object within the monocular endoscopic images as a function of the optical flow of the image point(s).

    Uncalibrated visual servoing using real-time velocity optimization
    23.
    发明授权
    Uncalibrated visual servoing using real-time velocity optimization 有权
    未校准的视觉伺服采用实时速度优化

    公开(公告)号:US08934003B2

    公开(公告)日:2015-01-13

    申请号:US13519383

    申请日:2010-11-17

    IPC分类号: H04N7/18 A61B19/00

    摘要: A robotic control method for a camera (30) having an optical view and a robot (40) having an end-effector (42) and one or more joints (41) for maneuvering end-effector (42). The robotic control method involves an acquisition of a digital video frame (32) illustrating an image as optically viewed by the camera (30), and an execution of a visual servoing for controlling a pose of end-effector (42) relative to an image feature within the digital video frame (32). The visual servoing involves an identification of a tracking vector (vtrk) within an image coordinate system (80) of the digital video frame (32) extending from a tracking point (TR) to a target point (TG) associated with the image feature, a mapping of the tracking vector within a configuration space (100) constructed from a robotic coordinate system (90) associated with the end-effector (42), and a derivation of a pose of the end-effector (42) within the robotic coordinate system (90) from the mapping of the tracking vector (vtrk) within the configuration space (100).

    摘要翻译: 一种用于具有光学视图的照相机(30)和具有端部执行器(42)的机器人(40)和用于操纵端部执行器(42)的一个或多个关节(41)的机器人控制方法。 机器人控制方法涉及一种数字视频帧(32)的采集,该数字视频帧(32)示出由照相机(30)光学观看的图像,以及执行用于控制相对于图像的末端执行器(42)的姿势的视觉伺服 特征在数字视频帧(32)内。 视觉伺服涉及从跟踪点(TR)延伸到与图像特征相关联的目标点(TG)的数字视频帧(32)的图像坐标系(80)内的跟踪矢量(vtrk)的识别, 在由与所述末端执行器(42)相关联的机器人坐标系(90)构造的配置空间(100)内的所述跟踪矢量的映射以及所述机器人坐标系中所述末端执行器(42)的姿态的推导 系统(90)从配置空间(100)内的跟踪矢量(vtrk)的映射。

    AUTOMATIC ONLINE REGISTRATION BETWEEN A ROBOT AND IMAGES
    24.
    发明申请
    AUTOMATIC ONLINE REGISTRATION BETWEEN A ROBOT AND IMAGES 有权
    机器人与图像之间的自动在线注册

    公开(公告)号:US20140212025A1

    公开(公告)日:2014-07-31

    申请号:US14342403

    申请日:2012-09-05

    IPC分类号: G06T3/00

    摘要: A registration system and method includes a configurable device (104) having one or more moveable features (122) such that movement of the moveable features can be determined relative to a reference to define a specific configuration of the configurable device. An imaging system (110) has a display on which the configurable device is viewable. A processing device (112) is configured to register the configurable device with a coordinate system of the imaging system based on the specific configuration of the configurable device.

    摘要翻译: 注册系统和方法包括具有一个或多个可移动特征(122)的可配置设备(104),使得可以相对于参考来确定可移动特征的移动以定义可配置设备的特定配置。 成像系统(110)具有可配置设备可见的显示器。 处理设备(112)被配置为基于可配置设备的具体配置向可配置设备注册成像系统的坐标系。

    ENDOSCOPY-GUIDED DEPLOYMENT OF VESSEL PUNCH
    25.
    发明申请
    ENDOSCOPY-GUIDED DEPLOYMENT OF VESSEL PUNCH 有权
    内窥镜引导血管穿刺的部署

    公开(公告)号:US20130325028A1

    公开(公告)日:2013-12-05

    申请号:US13877475

    申请日:2011-10-04

    IPC分类号: A61B19/00 A61B17/3205

    摘要: A robotic punch system employs a robot unit (70) and a control unit (80). The robot unit (70) includes a robot (71) and an endoscopic punch (72) mounted to the robot (71). The endoscopic punch (72) includes a calibrated spatial alignment of an endoscope (73) and a punch (74). The control unit (80) commands the robot (71) for deploying the endoscopic punch (72) in executing a puncture of an anatomical tissue (92).

    摘要翻译: 机器人打孔系统采用机器人单元(70)和控制单元(80)。 机器人单元(70)包括安装在机器人(71)上的机器人(71)和内窥镜冲头(72)。 内窥镜冲头(72)包括内窥镜(73)和冲头(74)的经校准的空间对准。 控制单元(80)命令机器人(71)用于在执行解剖组织(92)的穿刺时展开内窥镜冲头(72)。

    ROBOTIC CONTROL OF AN ENDOSCOPE FROM BLOOD VESSEL TREE IMAGES
    26.
    发明申请
    ROBOTIC CONTROL OF AN ENDOSCOPE FROM BLOOD VESSEL TREE IMAGES 有权
    从血管树图像中的内窥镜的机械控制

    公开(公告)号:US20130165948A1

    公开(公告)日:2013-06-27

    申请号:US13822001

    申请日:2011-09-13

    IPC分类号: A61B19/00

    摘要: A robot guiding system employs a robot unit (10) and a control unit (20). The robot unit (10) includes an endoscope (12) for generating an intra-operative endoscopic image (14) of a blood vessel tree within an anatomical region, and a robot (11) for moving the endoscope (12) within the anatomical region. The control unit (20) includes an endoscope controller (22) for generating an endoscopic path within the anatomical region, wherein the endoscopic path is derived from a matching of a graphical representation of the intra-operative endoscopic image (14) of the blood vessel tree to a graphical representation of a pre-operative three-dimensional image (44) of the blood vessel tree. The control unit (20) further includes a robot controller (21) for commanding the robot (11) to move the endoscope (12) within the anatomical region in accordance with the endoscopic path.

    摘要翻译: 机器人引导系统采用机器人单元(10)和控制单元(20)。 机器人单元(10)包括用于产生解剖区域内的血管树的手术内窥镜图像(14)的内窥镜(12)和用于在解剖区域内移动内窥镜(12)的机器人 。 控制单元(20)包括用于在解剖区域内产生内窥镜路径的内窥镜控制器(22),其中内窥镜路径是根据血管内手术内窥镜图像(14)的图形表示的匹配得到的 树到血管树的术前三维图像(44)的图形表示。 控制单元(20)还包括机器人控制器(21),用于根据内窥镜路径命令机器人(11)在解剖区域内移动内窥镜(12)。

    IMAGE INTEGRATION BASED REGISTRATION AND NAVIGATION FOR ENDOSCOPIC SURGERY
    27.
    发明申请
    IMAGE INTEGRATION BASED REGISTRATION AND NAVIGATION FOR ENDOSCOPIC SURGERY 有权
    基于图像整合的内镜手术注册和导航

    公开(公告)号:US20120294498A1

    公开(公告)日:2012-11-22

    申请号:US13521284

    申请日:2010-12-06

    IPC分类号: G06K9/36

    摘要: A real-time tracking of surgical tools relative to a pre-operative surgical plan and intra-operative images involves an image-based registration and tool tracking registration. The image-based registration implements an integration of a plurality of images of an anatomical region of a body including a pre-operative scan image (31) (e.g., a 3D CT/MRI image), an intra-operative fluoroscopic image (42) (e.g., an 2D X-ray image) and an intra- operative endoscopic image (23) (e.g., an 2D arthroscopic image). The tool tracking registration implements a representation within the pre-operative scan image (31) and/or the intra-operative fluoroscopic image (42) of a tracking of one or more surgical tools within the intra-operative endoscopic image (23).

    摘要翻译: 相对于术前手术计划和手术内图像的手术工具的实时跟踪涉及基于图像的注册和工具跟踪注册。 基于图像的注册实现了包括术前扫描图像(31)(例如,3D CT / MRI图像),手术内透视图像(42)的身体的解剖区域的多个图像的集成, (例如,2D X射线图像)和术中内窥镜图像(23)(例如,2D关节镜图像)。 工具跟踪注册实现术中内窥镜图像(23)内的一个或多个手术工具的跟踪的术前扫描图像(31)和/或手术内透视图像(42)内的表示。

    DISTANCE-BASED POSITION TRACKING METHOD AND SYSTEM
    28.
    发明申请
    DISTANCE-BASED POSITION TRACKING METHOD AND SYSTEM 审中-公开
    基于距离的位置跟踪方法和系统

    公开(公告)号:US20120063644A1

    公开(公告)日:2012-03-15

    申请号:US13321222

    申请日:2010-05-14

    IPC分类号: G06K9/00

    摘要: A pre-operative stage of a distance-based position tracking method (30) involves a generation of virtual information (21) derived from a scan image (20) illustrating an anatomical region (40) of a body during a virtual navigation of a surgical tool (51) relative to a surgical path (52) within scan image (20). The virtual information (21) includes a prediction of virtual poses of surgical tool (51) relative to surgical path (52) within scan image (20) associated with measurements of a virtual distance of surgical tool (51) from an object within scan image (20). An intra-operative stage of the method (30) involves a generation of tracking information (23) derived from measurements of a physical distance of surgical tool (51) from the object within anatomical region (40) during a physical navigation of surgical tool (51) relative to surgical path (52) within anatomical region (40). The tracking information (23) includes an estimation of poses of surgical tool (51) relative to surgical path (52) within anatomical region (40) corresponding to the prediction of virtual poses of surgical tool (51) relative to surgical path (52) within scan image (20).

    摘要翻译: 基于距离的位置跟踪方法(30)的手术前阶段涉及从扫描图像(20)导出的虚拟信息(21)的生成,其示出了在手术的虚拟导航期间身体的解剖区域(40) 工具(51)相对于扫描图像(20)内的外科路径(52)。 虚拟信息(21)包括与扫描图像(20)内的外科手术工具(51)的虚拟姿势的预测相关联的外科工具(51)的虚拟距离与扫描图像内的对象的测量 (20)。 所述方法(30)的手术中阶段涉及产生跟踪信息(23),所述跟踪信息(23)是在外科手术工具的物理导航期间从外科手术工具(51)与解剖学区域(40)内的物体的物理距离的测量得出的 在解剖区域(40)内相对于外科手术路径(52)。 跟踪信息(23)包括对应于相对于外科手术路径(52)预测外科手术工具(51)的虚拟姿势的解剖区域(40)内的外科手术工具(51)的姿势的估计, 在扫描图像(20)内。

    PLANNING AND ASSEMBLY OF COMPENSATING CONCENTRIC CANNULAS
    29.
    发明申请
    PLANNING AND ASSEMBLY OF COMPENSATING CONCENTRIC CANNULAS 审中-公开
    补充中枢性肛门的规划和组装

    公开(公告)号:US20110295199A1

    公开(公告)日:2011-12-01

    申请号:US13142456

    申请日:2009-11-10

    IPC分类号: A61M25/092 G06G7/48 G06G7/60

    摘要: A specification for a device including a set of concentric cannulas may be discovered to have an actual path different from a desired path, due to interactions between cannulas that effect net curvature of the device. The choice of particular cannulas may be corrected by performing a calculation taking into account curvature affecting properties of the individual cannulas including radius of curvature, elasticity, and moment of inertia. This calculation is preferably performed iteratively starting with a most distal cannula and iterating through the cannulas to the proximal end, accumulating net effect of curvature affecting properties.

    摘要翻译: 可以发现包括一组同心插管的装置的规格具有不同于期望路径的实际路径,这是由于影响装置的净曲率的插管之间的相互作用。 特定插管的选择可以通过考虑考虑包括曲率半径,弹性和惯性矩在内的各个插管的特性的曲率进行计算来校正。 该计算优选地以最远端插管开始迭代地迭代,并且通过套管迭代到近端,累积影响特性的曲率的净效应。

    Laser enhanced reconstruction of 3D surface

    公开(公告)号:US11022433B2

    公开(公告)日:2021-06-01

    申请号:US13577456

    申请日:2011-01-14

    摘要: A method for reconstructing a surface of a three-dimensional object involves a projection of a laser spot pattern onto the surface of the three-dimensional object by a laser, and a generation of a series of endoscopic images as an endoscope is translated and/or rotated relative to the three-dimensional object. Each endoscopic image illustrates a different view of a laser spot array within the laser spot pattern as projected onto the surface of the three-dimensional object by the laser. The laser spot array may be identical to or a subset of the laser spot pattern. The method further involves a reconstruction of the surface of the three-dimensional object from a correspondence of the different views of the laser spot array as illustrated in the endoscopic images.