公开(公告)号：US08830224B2

公开(公告)日：2014-09-09

申请号：US12465020

申请日：2009-05-13

申请人： Wenyi Zhao ,  Chenyu Wu ,  David Hirvonen ,  Christopher J. Hasser ,  Brian E. Miller ,  Catherine J. Mohr ,  Myrian J. Curet ,  Tao Zhao ,  Simon Di Maio ,  Brian D. Hoffman

发明人： Wenyi Zhao ,  Chenyu Wu ,  David Hirvonen ,  Christopher J. Hasser ,  Brian E. Miller ,  Catherine J. Mohr ,  Myrian J. Curet ,  Tao Zhao ,  Simon Di Maio ,  Brian D. Hoffman

IPC分类号： G06T15/00 ,  G06K9/68

CPC分类号： A61B34/25 ,  A61B34/20 ,  A61B2034/2057 ,  A61B2034/2065 ,  A61B2090/371 ,  A61B2090/373 ,  G06T7/593 ,  G06T11/00 ,  G06T2207/10068 ,  G06T2207/20096 ,  G06T2207/20101 ,  G06T2207/30004 ,  H04N13/10 ,  H04N13/20 ,  H04N13/30 ,  H04N2013/0092

摘要： An apparatus is configured to show telestration in 3-D to a surgeon in real time. A proctor is shown one side of a stereo image pair, such that the proctor can draw a telestration line on the one side with an input device. Points of interest are identified for matching to the other side of the stereo image pair. In response to the identified points of interest, regions and features are identified and used to match the points of interest to the other side. Regions can be used to match the points of interest. Features of the first image can be matched to the second image and used to match the points of interest to the second image, for example when the confidence scores for the regions are below a threshold value. Constraints can be used to evaluate the matched points of interest, for example by excluding bad points.

摘要翻译： 一种装置被配置为实时地向外科医生示出3-D的远程化。 监视器被显示为立体图像对的一侧，使得监视器可以在一侧用输入装置绘制远距离信号线。 识别兴趣点以匹配立体图像对的另一侧。 响应于所识别的兴趣点，区域和特征被识别并用于将感兴趣点与另一侧匹配。 区域可用于匹配兴趣点。 第一图像的特征可以匹配第二图像，并且用于将感兴趣的点与第二图像匹配，例如当区域的置信度得分低于阈值时。 限制可用于评估匹配的兴趣点，例如排除不良点。

公开(公告)号：US08184880B2

公开(公告)日：2012-05-22

申请号：US12465029

申请日：2009-05-13

申请人： Wenyi Zhao ,  Chenyu Wu ,  David Hirvonen ,  Tao Zhao ,  Brian D. Hoffman ,  Simon Di Maio

发明人： Wenyi Zhao ,  Chenyu Wu ,  David Hirvonen ,  Tao Zhao ,  Brian D. Hoffman ,  Simon Di Maio

IPC分类号： G06T7/00

CPC分类号： G06K9/6211 ,  G06T7/33

摘要： Systems, methods, and devices are used to match images. Points of interest from a first image are identified for matching to a second image. In response to the identified points of interest, regions and features can be identified and used to match the points of interest to a corresponding second image or second series of images. Regions can be used to match the points of interest when regions of the first image are matched to the second image with high confidence scores, for example above a threshold. Features of the first image can be matched to the second image, and these matched features may be used to match the points of interest to the second image, for example when the confidence scores for the regions are below the threshold value. Constraint can be used to evaluate the matched points of interest, for example by excluding bad points.

摘要翻译： 系统，方法和设备用于匹配图像。 识别来自第一图像的兴趣点以匹配第二图像。 响应于所识别的兴趣点，区域和特征可被识别并用于将感兴趣点与对应的第二图像或第二系列图像相匹配。 当第一图像的区域与具有高置信度得分的第二图像匹配时，例如高于阈值，可以使用区域来匹配感兴趣点。 第一图像的特征可以匹配第二图像，并且这些匹配的特征可以用于将感兴趣的点与第二图像匹配，例如当区域的置信度得分低于阈值时。 约束可用于评估匹配的兴趣点，例如排除不良点。

公开(公告)号：US08639000B2

公开(公告)日：2014-01-28

申请号：US13454297

申请日：2012-04-24

申请人： Wenyi Zhao ,  Chenyu Wu ,  David Hirvonen ,  Tao Zhao ,  Brian D. Hoffman ,  Simon Di Maio

发明人： Wenyi Zhao ,  Chenyu Wu ,  David Hirvonen ,  Tao Zhao ,  Brian D. Hoffman ,  Simon Di Maio

IPC分类号： G06T7/00

CPC分类号： G06K9/6211 ,  G06T7/33

摘要： Systems, methods, and devices are used to match images. Points of interest from a first image are identified for matching to a second image. In response to the identified points of interest, regions and features can be identified and used to match the points of interest to a corresponding second image or second series of images. Regions can be used to match the points of interest when regions of the first image are matched to the second image with high confidence scores, for example above a threshold. Features of the first image can be matched to the second image, and these matched features may be used to match the points of interest to the second image, for example when the confidence scores for the regions are below the threshold value. Constraint can be used to evaluate the matched points of interest, for example by excluding bad points.

摘要翻译： 系统，方法和设备用于匹配图像。 识别来自第一图像的兴趣点以匹配第二图像。 响应于所识别的兴趣点，区域和特征可被识别并用于将感兴趣点与对应的第二图像或第二系列图像相匹配。 当第一图像的区域与具有高置信度得分的第二图像匹配时，例如高于阈值，可以使用区域来匹配感兴趣点。 第一图像的特征可以匹配第二图像，并且这些匹配的特征可以用于将感兴趣的点与第二图像匹配，例如当区域的置信度得分低于阈值时。 约束可用于评估匹配的兴趣点，例如排除不良点。

公开(公告)号：US20100166323A1

公开(公告)日：2010-07-01

申请号：US12465029

申请日：2009-05-13

申请人： Wenyi Zhao ,  Chenyu Wu ,  David Hirvonen ,  Tao Zhao ,  Brian D. Hoffman ,  Simon Dimaio

发明人： Wenyi Zhao ,  Chenyu Wu ,  David Hirvonen ,  Tao Zhao ,  Brian D. Hoffman ,  Simon Dimaio

IPC分类号： G06K9/68

CPC分类号： G06K9/6211 ,  G06T7/33

摘要： Systems, methods, and devices are used to match images. Points of interest from a first image are identified for matching to a second image. In response to the identified points of interest, regions and features can be identified and used to match the points of interest to a corresponding second image or second series of images. Regions can be used to match the points of interest when regions of the first image are matched to the second image with high confidence scores, for example above a threshold. Features of the first image can be matched to the second image, and these matched features may be used to match the points of interest to the second image, for example when the confidence scores for the regions are below the threshold value. Constraint can be used to evaluate the matched points of interest, for example by excluding bad points.

摘要翻译： 系统，方法和设备用于匹配图像。 识别来自第一图像的兴趣点以匹配第二图像。 响应于所识别的兴趣点，区域和特征可被识别并用于将感兴趣点与对应的第二图像或第二系列图像相匹配。 当第一图像的区域与具有高置信度得分的第二图像匹配时，例如高于阈值，可以使用区域来匹配感兴趣点。 第一图像的特征可以匹配第二图像，并且这些匹配的特征可以用于将感兴趣的点与第二图像匹配，例如当区域的置信度得分低于阈值时。 约束可用于评估匹配的兴趣点，例如排除不良点。

公开(公告)号：US20100164950A1

公开(公告)日：2010-07-01

申请号：US12465020

申请日：2009-05-13

申请人： Wenyi Zhao ,  Chenyu Wu ,  David Hirvonen ,  Christopher J. Hasser ,  Brian E. Miller ,  Catherine J. Mohr ,  Myriam J. Curet ,  Tao Zhao ,  Simon Dimaio ,  Brian D. Hoffman

发明人： Wenyi Zhao ,  Chenyu Wu ,  David Hirvonen ,  Christopher J. Hasser ,  Brian E. Miller ,  Catherine J. Mohr ,  Myriam J. Curet ,  Tao Zhao ,  Simon Dimaio ,  Brian D. Hoffman

IPC分类号： G06T15/00

CPC分类号： A61B34/25 ,  A61B34/20 ,  A61B2034/2057 ,  A61B2034/2065 ,  A61B2090/371 ,  A61B2090/373 ,  G06T7/593 ,  G06T11/00 ,  G06T2207/10068 ,  G06T2207/20096 ,  G06T2207/20101 ,  G06T2207/30004 ,  H04N13/10 ,  H04N13/20 ,  H04N13/30 ,  H04N2013/0092

摘要： An apparatus is configured to show telestration in 3-D to a surgeon in real time. A proctor is shown one side of a stereo image pair, such that the proctor can draw a telestration line on the one side with an input device. Points of interest are identified for matching to the other side of the stereo image pair. In response to the identified points of interest, regions and features are identified and used to match the points of interest to the other side. Regions can be used to match the points of interest. Features of the first image can be matched to the second image and used to match the points of interest to the second image, for example when the confidence scores for the regions are below a threshold value. Constraints can be used to evaluate the matched points of interest, for example by excluding bad points.

摘要翻译： 一种装置被配置为实时地向外科医生示出3-D的远程化。 监视器被显示为立体图像对的一侧，使得监视器可以在一侧用输入装置绘制远距离信号线。 识别兴趣点以匹配立体图像对的另一侧。 响应于所识别的兴趣点，区域和特征被识别并用于将感兴趣点与另一侧匹配。 区域可用于匹配兴趣点。 第一图像的特征可以匹配第二图像，并且用于将感兴趣的点与第二图像匹配，例如当区域的置信度得分低于阈值时。 限制可用于评估匹配的兴趣点，例如排除不良点。

公开(公告)号：US09492240B2

公开(公告)日：2016-11-15

申请号：US12485503

申请日：2009-06-16

申请人： Brandon D. Itkowitz ,  Tao Zhao ,  Simon Di Maio ,  Wenyi Zhao ,  Christopher J. Hasser ,  Myrian J. Curet ,  Catherine J. Mohr ,  Hubert Stein

发明人： Brandon D. Itkowitz ,  Tao Zhao ,  Simon Di Maio ,  Wenyi Zhao ,  Christopher J. Hasser ,  Myrian J. Curet ,  Catherine J. Mohr ,  Hubert Stein

IPC分类号： A61B17/00 ,  A61B1/04 ,  A61B1/00

CPC分类号： A61B90/06 ,  A61B1/00193 ,  A61B34/25 ,  A61B34/30 ,  A61B34/35 ,  A61B34/37 ,  A61B90/361 ,  A61B2017/00477 ,  A61B2034/2065 ,  A61B2090/061

摘要： Robotic and/or measurement devices, systems, and methods for telesurgical and other applications employ input devices operatively coupled to tools so as to allow a system user to manipulate tissues and other structures being measured. The system may make use of three dimensional position information from stereoscopic images. Two or more discrete points can be designated in three dimensions so as to provide a cumulative length along a straight or curving structure, an area measurement, a volume measurement, or the like. The discrete points may be identified by a single surgical tool or by distances separating two or more surgical tools, with the user optionally measuring a structure longer than a field of view of the stereoscopic image capture device by walking a pair of tools “hand-over-hand” along the structure. By allowing the system user to interact with the tissues while designating the tissue locations, and by employing imaging data to determine the measurements, the measurement accuracy and ease of measurement may be enhanced.

公开(公告)号：US09526587B2

公开(公告)日：2016-12-27

申请号：US12428657

申请日：2009-04-23

申请人： Tao Zhao ,  Wenyi Zhao ,  Daniel J. Halabe ,  Brian D. Hoffman ,  William C. Nowlin

发明人： Tao Zhao ,  Wenyi Zhao ,  Daniel J. Halabe ,  Brian D. Hoffman ,  William C. Nowlin

IPC分类号： A61B5/05

CPC分类号： A61B34/30 ,  A61B34/37 ,  A61B90/94 ,  A61B2034/2055 ,  A61B2034/2065 ,  A61B2090/3937 ,  A61B2090/3983

摘要： The present disclosure relates to systems, methods, and tools for tool tracking using image-derived data from one or more tool-located reference features. A method includes: capturing a first image of a tool that includes multiple features that define a first marker, where at least one of the features of the first marker includes an identification feature; determining a position for the first marker by processing the first image; determining an identification for the first marker by using the at least one identification feature by processing the first image; and determining a tool state for the tool by using the position and the identification of the first marker.

摘要翻译： 本公开涉及使用来自一个或多个位于工具的参考特征的图像衍生数据进行工具跟踪的系统，方法和工具。 一种方法包括：捕获包括定义第一标记的多个特征的工具的第一图像，其中第一标记的至少一个特征包括识别特征; 通过处理第一图像确定第一标记的位置; 通过处理所述第一图像来使用所述至少一个识别特征来确定所述第一标记的识别; 以及通过使用所述位置和所述第一标记的识别来确定所述工具的工具状态。

公开(公告)号：US08169468B2

公开(公告)日：2012-05-01

申请号：US12164363

申请日：2008-06-30

申请人： David Scott ,  Wenyi Zhao ,  Christopher J. Hasser ,  Brian D. Hoffman ,  Paul Lilagan ,  Ian McDowall ,  Catherine J. Mohr ,  John D. Stern ,  Tao Zhao

发明人： David Scott ,  Wenyi Zhao ,  Christopher J. Hasser ,  Brian D. Hoffman ,  Paul Lilagan ,  Ian McDowall ,  Catherine J. Mohr ,  John D. Stern ,  Tao Zhao

IPC分类号： H04N13/00 ,  G06F12/00

CPC分类号： A61B1/043 ,  A61B1/00009 ,  A61B1/0005 ,  A61B1/00149 ,  A61B1/00186 ,  A61B1/00193 ,  A61B1/05 ,  A61B1/063 ,  A61B1/0638 ,  A61B1/313 ,  A61B5/0071 ,  A61B5/0075 ,  A61B5/0084 ,  A61B34/30 ,  A61B34/37 ,  A61B90/30 ,  A61B90/361 ,  A61B2034/301 ,  A61B2090/364 ,  G01N21/6456 ,  H04N13/0239 ,  H04N13/207 ,  H04N13/239

摘要： A robotic surgical system positions and holds an endoscope. A visible imaging system is coupled to the endoscope. The visible imaging system captures a visible image of tissue. An alternate imaging system is also coupled to the endoscope. The alternate imaging system captures a fluorescence image of at least a portion of the tissue. A stereoscopic video display system is coupled to the visible imaging system and to the alternate imaging system. The stereoscopic video display system outputs a real-time stereoscopic image comprising a three-dimensional presentation of a blend of a fluorescence image associated with the captured fluorescence image, and the visible image.

摘要翻译： 机器人手术系统定位并保持内窥镜。 可见成像系统连接到内窥镜。 可见成像系统捕获组织的可见图像。 替代的成像系统也耦合到内窥镜。 替代成像系统捕获组织的至少一部分的荧光图像。 立体视频显示系统耦合到可见成像系统和替代成像系统。 立体视频显示系统输出包括与所捕获的荧光图像相关联的荧光图像的混合物和可见图像的三维呈现的实时立体图像。

公开(公告)号：US08228368B2

公开(公告)日：2012-07-24

申请号：US12164976

申请日：2008-06-30

申请人： Wenyi Zhao ,  Catherine J. Mohr ,  Christopher J. Hasser ,  Brian D. Hoffman ,  John D. Stern ,  Tao Zhao

发明人： Wenyi Zhao ,  Catherine J. Mohr ,  Christopher J. Hasser ,  Brian D. Hoffman ,  John D. Stern ,  Tao Zhao

IPC分类号： H04N13/00

CPC分类号： A61B1/043 ,  A61B1/00009 ,  A61B1/0005 ,  A61B1/00149 ,  A61B1/00186 ,  A61B1/00193 ,  A61B1/05 ,  A61B1/063 ,  A61B1/0638 ,  A61B1/313 ,  A61B5/0071 ,  A61B5/0075 ,  A61B5/0084 ,  A61B34/30 ,  A61B34/37 ,  A61B90/30 ,  A61B90/361 ,  A61B2034/301 ,  A61B2090/364 ,  G01N21/6456 ,  H04N13/0239 ,  H04N13/207 ,  H04N13/239

摘要： An illumination channel, a stereoscopic optical channel and another optical channel are held and positioned by a robotic surgical system. A first capture unit captures a stereoscopic visible image from the first light from the stereoscopic optical channel while a second capture unit captures a fluorescence image from the second light from the other optical channel. An intelligent image processing system receives the captured stereoscopic visible image and the captured fluorescence image and generates a stereoscopic pair of fluorescence images. An augmented stereoscopic display system outputs a real-time stereoscopic image comprising a three-dimensional presentation of a blend of the stereoscopic visible image and the stereoscopic pair of fluorescence images.

摘要翻译： 照明通道，立体光学通道和另一个光学通道由机器人手术系统保持和定位。 第一捕获单元从来自立体光学信道的第一光捕获立体可视图像，而第二捕获单元从来自另一光信道的第二光捕获荧光图像。 智能图像处理系统接收捕获的立体可视图像和捕获的荧光图像并产生立体对的荧光图像。 增强立体显示系统输出包括立体可视图像和立体视觉对荧光图像的混合的三维呈现的实时立体图像。

公开(公告)号：US20090268012A1

公开(公告)日：2009-10-29

申请号：US12165194

申请日：2008-06-30

申请人： David Scott ,  Wenyi Zhao ,  Brian D. Hoffman ,  John D. Stern ,  Tao Zhao

发明人： David Scott ,  Wenyi Zhao ,  Brian D. Hoffman ,  John D. Stern ,  Tao Zhao

IPC分类号： H04N13/04 ,  H04N7/18 ,  H04N15/00 ,  A61B1/04

CPC分类号： A61B1/043 ,  A61B1/00009 ,  A61B1/0005 ,  A61B1/00149 ,  A61B1/00186 ,  A61B1/00193 ,  A61B1/05 ,  A61B1/063 ,  A61B1/0638 ,  A61B1/313 ,  A61B5/0071 ,  A61B5/0075 ,  A61B5/0084 ,  A61B34/30 ,  A61B34/37 ,  A61B90/30 ,  A61B90/361 ,  A61B2034/301 ,  A61B2090/364 ,  G01N21/6456 ,  H04N13/0239 ,  H04N13/207 ,  H04N13/239

摘要： An endoscope with a stereoscopic optical channel is held and positioned by a robotic surgical system. A capture unit captures (1) a visible first image and (2) a visible second image combined with a fluorescence second image from the light. An intelligent image processing system receives (1) the visible first image and (2) the visible second image combined with the fluorescence second image and generates at least one fluorescence image of a stereoscopic pair of fluorescence images and a visible second image. An augmented stereoscopic display system outputs a real-time stereoscopic image including a three-dimensional presentation including in one eye, a blend of the at least one fluorescence image of a stereoscopic pair of fluorescence images and one of the visible first and second images; and in the other eye, the other of the visible first and second images.

摘要翻译： 具有立体光学通道的内窥镜由机器人手术系统保持和定位。 捕获单元捕获（1）可见的第一图像和（2）与来自光的荧光第二图像组合的可见第二图像。 智能图像处理系统接收（1）可见第一图像和（2）与荧光第二图像组合的可见第二图像，并产生立体对荧光图像和可见第二图像的至少一个荧光图像。 增强立体显示系统输出包括在一只眼睛中的三维呈现的实时立体图像，立体对荧光图像对的至少一个荧光图像与可见第一和第二图像之一的混合; 在另一只眼睛中，可见的第一和第二图像中的另一个。