公开(公告)号：US07587516B2

公开(公告)日：2009-09-08

申请号：US10468999

申请日：2002-02-25

申请人： Gyan Bhanot ,  Matthias A. Blumrich ,  Dong Chen ,  Paul W. Coteus ,  Alan G. Gara ,  Mark E. Giampapa ,  Philip Heidelberger ,  Burkhard D. Steinmacher-Burow ,  Todd E. Takken ,  Pavlos M. Vranas

发明人： Gyan Bhanot ,  Matthias A. Blumrich ,  Dong Chen ,  Paul W. Coteus ,  Alan G. Gara ,  Mark E. Giampapa ,  Philip Heidelberger ,  Burkhard D. Steinmacher-Burow ,  Todd E. Takken ,  Pavlos M. Vranas

IPC分类号： G06F15/173 ,  H04L12/66 ,  H04L12/50

CPC分类号： H04L45/16 ,  H04L45/06

摘要： Class network routing is implemented in a network such as a computer network comprising a plurality of parallel compute processors at nodes thereof. Class network routing allows a compute processor to broadcast a message to a range (one or more) of other compute processors in the computer network, such as processors in a column or a row. Normally this type of operation requires a separate message to be sent to each processor. With class network routing pursuant to the invention, a single message is sufficient, which generally reduces the total number of messages in the network as well as the latency to do a broadcast. Class network routing is also applied to dense matrix inversion algorithms on distributed memory parallel supercomputers with hardware class function (multicast) capability. This is achieved by exploiting the fact that the communication patterns of dense matrix inversion can be served by hardware class functions, which results in faster execution times.

摘要翻译： 在诸如包括在其节点处的多个并行计算处理器的计算机网络的网络中实现类网络路由。 类网络路由允许计算处理器将消息广播到计算机网络中的其他计算处理器的范围（一个或多个），例如列或行中的处理器。 通常这种类型的操作需要单独的消息发送到每个处理器。 根据本发明的类网络路由，单个消息是足够的，这通常减少了网络中的消息总数以及进行广播的延迟。 类网络路由也适用于具有硬件类功能（组播）能力的分布式存储并行超级计算机上的密集矩阵求逆算法。 这是通过利用密集矩阵反演的通信模式可以通过硬件类功能来实现的，这导致更快的执行时间。

公开(公告)号：US07330996B2

公开(公告)日：2008-02-12

申请号：US10468990

申请日：2002-02-25

申请人： Paul W. Coteus ,  Alan G. Gara ,  Mark E. Giampapa ,  Philip Heidelberger ,  Burkhard D. Steinmacher-Burow

发明人： Paul W. Coteus ,  Alan G. Gara ,  Mark E. Giampapa ,  Philip Heidelberger ,  Burkhard D. Steinmacher-Burow

IPC分类号： G06F11/00 ,  G06F11/20

CPC分类号： H05K7/20836 ,  F24F11/77 ,  G06F9/52 ,  G06F9/526 ,  G06F15/17381 ,  G06F17/142 ,  G09G5/008 ,  H04L7/0338

摘要： A method for maintaining full performance of a file system in the presence of a failure is provided. The file system having N storage devices, where N is an integer greater than zero and N primary file servers where each file server is operatively connected to a corresponding storage device for accessing files therein. The file system further having a secondary file server operatively connected to at least one of the N storage devices. The method including: switching the connection of one of the N storage devices to the secondary file server upon a failure of one of the N primary file servers; and switching the connections of one or more of the remaining storage devices to a primary file server other than the failed file server as necessary so as to prevent a loss in performance and to provide each storage device with an operating file server.

摘要翻译： 提供了在出现故障的情况下维持文件系统的完整性能的方法。 具有N个存储设备的文件系统，其中N是大于零的整数，N个主文件服务器，其中每个文件服务器可操作地连接到用于访问其中的文件的相应存储设备。 所述文件系统还具有可操作地连接到所述N个存储设备中的至少一个的辅助文件服务器。 该方法包括：在N个主要文件服务器之一发生故障时，将N个存储设备之一的连接切换到次要文件服务器; 并且根据需要将一个或多个剩余存储设备的连接切换到除故障文件服务器之外的主文件服务器，以防止性能损失并向每个存储设备提供操作文件服务器。

公开(公告)号：US20090006296A1

公开(公告)日：2009-01-01

申请号：US11768795

申请日：2007-06-26

申请人： Dong Chen ,  Alan G. Gara ,  Mark E. Giampapa ,  Philip Heidelberger ,  Burkhard Steinmacher-Burow ,  Pavlos Vranas

发明人： Dong Chen ,  Alan G. Gara ,  Mark E. Giampapa ,  Philip Heidelberger ,  Burkhard Steinmacher-Burow ,  Pavlos Vranas

IPC分类号： G06F15/18

CPC分类号： G06F15/163

摘要： A parallel computer system is constructed as a network of interconnected compute nodes to operate a global message-passing application for performing communications across the network. Each of the compute nodes includes one or more individual processors with memories which run local instances of the global message-passing application operating at each compute node to carry out local processing operations independent of processing operations carried out at other compute nodes. Each compute node also includes a DMA engine constructed to interact with the application via Injection FIFO Metadata describing multiple Injection FIFOs where each Injection FIFO may containing an arbitrary number of message descriptors in order to process messages with a fixed processing overhead irrespective of the number of message descriptors included in the Injection FIFO.

摘要翻译： 并行计算机系统被构造为互连的计算节点的网络，以操作用于在整个网络上执行通信的全局消息传递应用。 每个计算节点包括具有存储器的一个或多个单独处理器，该存储器运行在每个计算节点处操作的全局消息传递应用的本地实例，以独立于在其他计算节点执行的处理操作来执行本地处理操作。 每个计算节点还包括构造成通过描述多个注入FIFO的注入FIFO元数据与应用交互的DMA引擎，其中每个注入FIFO可以包含任意数量的消息描述符，以便处理具有固定处理开销的消息，而不管消息的数量 描述符包含在注入FIFO中。

公开(公告)号：US06895416B2

公开(公告)日：2005-05-17

申请号：US10258515

申请日：2002-02-25

申请人： Alan G. Gara ,  Mark E. Giampapa ,  Burkhard D. Steinmacher-Burow

发明人： Alan G. Gara ,  Mark E. Giampapa ,  Burkhard D. Steinmacher-Burow

IPC分类号： G06F7/00 ,  G06F17/00

CPC分类号： G06F11/1438 ,  Y10S707/99953 ,  Y10S707/99955

摘要： The present in invention is directed to a checkpointing filesystem of a distributed-memory parallel supercomputer comprising a node that accesses user data on the filesystem, the filesystem comprising an interface that is associated with a disk for storing the user data. The checkpointing filesystem provides for taking and checkpoint of the filesystem and rolling back to a previously taken checkpoint, as well as for writing user data to and deleting user data from the checkpointing filesystem. The checkpointing filesystem provides a recently written file allocation table (WFAT) for maintaining information regarding the user data written since a previously taken checkpoint and a recently deleted file allocation table (DFAT) for maintaining information regarding user data deleted from since the previously taken checkpoint, both of which are utilized by the checkpointing filesystem to take a checkpoint of the filesystem and rollback the filesystem to a previously taken checkpoint, as well as to write and delete user data from the checkpointing filesystem.

摘要翻译： 本发明涉及一种分布式存储器并行超级计算机的检查点文件系统，其包括访问文件系统上的用户数据的节点，该文件系统包括与用于存储用户数据的盘相关联的接口。 检查点文件系统提供文件系统的获取和检查点，并回滚到先前执行的检查点，以及从检查点文件系统向用户数据写入和删除用户数据。 检查点文件系统提供最近写入的文件分配表（WFAT），用于维护关于自先前检查点以来写入的用户数据的信息，以及用于维护关于从先前检查点以来删除的用户数据的信息的最近删除的文件分配表（DFAT） 这两个都由检查点文件系统用于采取文件系统的检查点并将文件系统回滚到先前执行的检查点，以及从检查点文件系统写入和删除用户数据。

公开(公告)号：US20120311299A1

公开(公告)日：2012-12-06

申请号：US13566024

申请日：2012-08-03

申请人： Matthias A. Blumrich ,  Dong Chen ,  George L. Chiu ,  Thomas M. Cipolla ,  Paul W. Coteus ,  Alan G. Gara ,  Mark E. Giampapa ,  Philip Heidlberger ,  Gerard V. Kopcsay ,  Lawrence S. Mok ,  Todd E. Takken

发明人： Matthias A. Blumrich ,  Dong Chen ,  George L. Chiu ,  Thomas M. Cipolla ,  Paul W. Coteus ,  Alan G. Gara ,  Mark E. Giampapa ,  Philip Heidlberger ,  Gerard V. Kopcsay ,  Lawrence S. Mok ,  Todd E. Takken

IPC分类号： G06F15/80

CPC分类号： H05K7/20836 ,  F24F11/77 ,  G06F9/52 ,  G06F9/526 ,  G06F15/17381 ,  G06F17/142 ,  G09G5/008 ,  H04L7/0338

摘要： A novel massively parallel supercomputer of hundreds of teraOPS-scale includes node architectures based upon System-On-a-Chip technology, i.e., each processing node comprises a single Application Specific Integrated Circuit (ASIC). Within each ASIC node is a plurality of processing elements each of which consists of a central processing unit (CPU) and plurality of floating point processors to enable optimal balance of computational performance, packaging density, low cost, and power and cooling requirements. The plurality of processors within a single node individually or simultaneously work on any combination of computation or communication as required by the particular algorithm being solved. The system-on-a-chip ASIC nodes are interconnected by multiple independent networks that optimally maximizes packet communications throughput and minimizes latency. The multiple networks include three high-speed networks for parallel algorithm message passing including a Torus, Global Tree, and a Global Asynchronous network that provides global barrier and notification functions.

摘要翻译： 数百个teraOPS级别的新型大规模并行超级计算机包括基于片上系统技术的节点架构，即每个处理节点包括单个专用集成电路（ASIC）。 在每个ASIC节点内是多个处理元件，每个处理元件由中央处理单元（CPU）和多个浮点处理器组成，以实现计算性能，封装密度，低成本以及功率和冷却​​要求的最佳平衡。 单个节点内的多个处理器单独或同时工作在要解决的特定算法所要求的计算或通信的任何组合上。 片上系统ASIC节点通过多个独立网络进行互连，从而最大限度地最大限度地提高了分组通信吞吐量并最大限度地减少了延迟。 多个网络包括用于并行算法消息传递的三个高速网络，包括Torus，全局树和提供全局障碍和通知功能的全球异步网络。

公开(公告)号：US08250133B2

公开(公告)日：2012-08-21

申请号：US12492799

申请日：2009-06-26

申请人： Matthias A. Blumrich ,  Dong Chen ,  George L. Chiu ,  Thomas M. Cipolla ,  Paul W. Coteus ,  Alan G. Gara ,  Mark E. Giampapa ,  Philip Heidelberger ,  Gerard V. Kopcsay ,  Lawrence S. Mok ,  Todd E. Takken

发明人： Matthias A. Blumrich ,  Dong Chen ,  George L. Chiu ,  Thomas M. Cipolla ,  Paul W. Coteus ,  Alan G. Gara ,  Mark E. Giampapa ,  Philip Heidelberger ,  Gerard V. Kopcsay ,  Lawrence S. Mok ,  Todd E. Takken

IPC分类号： G06F15/16

CPC分类号： H05K7/20836 ,  F24F11/77 ,  G06F9/52 ,  G06F9/526 ,  G06F15/17381 ,  G06F17/142 ,  G09G5/008 ,  H04L7/0338

摘要： A novel massively parallel supercomputer of hundreds of teraOPS-scale includes node architectures based upon System- On-a-Chip technology, i.e., each processing node comprises a single Application Specific Integrated Circuit (ASIC). Within each ASIC node is a plurality of processing elements each of which consists of a central processing unit (CPU) and plurality of floating point processors to enable optimal balance of computational performance, packaging density, low cost, and power and cooling requirements. The plurality of processors within a single node individually or simultaneously work on any combination of computation or communication as required by the particular algorithm being solved. The system-on-a-chip ASIC nodes are interconnected by multiple independent networks that optimally maximizes packet communications throughput and minimizes latency. The multiple networks include three high-speed networks for parallel algorithm message passing including a Torus, Global Tree, and a Global Asynchronous network that provides global barrier and notification functions.

摘要翻译： 数百个teraOPS级别的新型大规模并行超级计算机包括基于片上系统技术的节点架构，即每个处理节点包括单个专用集成电路（ASIC）。 在每个ASIC节点内是多个处理元件，每个处理元件由中央处理单元（CPU）和多个浮点处理器组成，以实现计算性能，封装密度，低成本以及功率和冷却​​要求的最佳平衡。 单个节点内的多个处理器单独或同时工作在要解决的特定算法所要求的计算或通信的任何组合上。 片上系统ASIC节点通过多个独立网络互连，从而最大限度地最大限度地提高了分组通信吞吐量并最大限度地减少了延迟。 多个网络包括用于并行算法消息传递的三个高速网络，包括Torus，全局树和提供全局障碍和通知功能的全球异步网络。

公开(公告)号：US07555566B2

公开(公告)日：2009-06-30

申请号：US10468993

申请日：2002-02-25

申请人： Matthias A. Blumrich ,  Dong Chen ,  George L. Chiu ,  Thomas M. Cipolla ,  Paul W. Coteus ,  Alan G. Gara ,  Mark E. Giampapa ,  Philip Heidelberger ,  Gerard V. Kopcsay ,  Lawrence S. Mok ,  Todd E. Takken

发明人： Matthias A. Blumrich ,  Dong Chen ,  George L. Chiu ,  Thomas M. Cipolla ,  Paul W. Coteus ,  Alan G. Gara ,  Mark E. Giampapa ,  Philip Heidelberger ,  Gerard V. Kopcsay ,  Lawrence S. Mok ,  Todd E. Takken

IPC分类号： G06F15/16

CPC分类号： H05K7/20836 ,  F24F11/77 ,  G06F9/52 ,  G06F9/526 ,  G06F15/17381 ,  G06F17/142 ,  G09G5/008 ,  H04L7/0338

摘要： A novel massively parallel supercomputer of hundreds of teraOPS-scale includes node architectures based upon System-On-a-Chip technology, i.e., each processing node comprises a single Application Specific Integrated Circuit (ASIC). Within each ASIC node is a plurality of processing elements each of which consists of a central processing unit (CPU) and plurality of floating point processors to enable optimal balance of computational performance, packaging density, low cost, and power and cooling requirements. The plurality of processors within a single node may be used individually or simultaneously to work on any combination of computation or communication as required by the particular algorithm being solved or executed at any point in time. The system-on-a-chip ASIC nodes are interconnected by multiple independent networks that optimally maximizes packet communications throughput and minimizes latency. In the preferred embodiment, the multiple networks include three high-speed networks for parallel algorithm message passing including a Torus, Global Tree, and a Global Asynchronous network that provides global barrier and notification functions. These multiple independent networks may be collaboratively or independently utilized according to the needs or phases of an algorithm for optimizing algorithm processing performance. For particular classes of parallel algorithms, or parts of parallel calculations, this architecture exhibits exceptional computational performance, and may be enabled to perform calculations for new classes of parallel algorithms. Additional networks are provided for external connectivity and used for Input/Output, System Management and Configuration, and Debug and Monitoring functions. Special node packaging techniques implementing midplane and other hardware devices facilitates partitioning of the supercomputer in multiple networks for optimizing supercomputing resources.

摘要翻译： 数百个teraOPS级别的新型大规模并行超级计算机包括基于片上系统技术的节点架构，即，每个处理节点包括单个专用集成电路（ASIC）。 在每个ASIC节点内是多个处理元件，每个处理元件由中央处理单元（CPU）和多个浮点处理器组成，以实现计算性能，封装密度，低成本以及功率和冷却​​要求的最佳平衡。 单个节点内的多个处理器可以单独使用或同时使用，以在任何时间点解决或执行的特定算法所要求的任何计算或通信组合上工作。 片上系统ASIC节点通过多个独立网络互连，从而最大限度地最大限度地提高了分组通信吞吐量并最大限度地减少了延迟。 在优选实施例中，多个网络包括用于并行算法消息传递的三个高速网络，包括提供全局障碍和通知功能的环形，全局树和全球异步网络。 这些多个独立网络可以根据用于优化算法处理性能的算法的需求或阶段来协同或独立地利用。 对于特定类别的并行算法或并行计算的部分，该架构具有出色的计算性能，并且可以启用对新类并行算法执行计算。 为外部连接提供附加网络，用于输入/输出，系统管理和配置以及调试和监控功能。 实现中平面和其他硬件设备的特殊节点打包技术有助于在多个网络中划分超级计算机，以优化超级计算资源。

公开(公告)号：US07305487B2

公开(公告)日：2007-12-04

申请号：US10469001

申请日：2002-02-25

申请人： Matthias A. Blumrich ,  Dong Chen ,  Paul W. Coteus ,  Alan G. Gara ,  Mark E. Giampapa ,  Philip Heidelberger ,  Burkhard D. Steinmacher-Burow ,  Todd E. Takken ,  Pavlos M. Vranas

发明人： Matthias A. Blumrich ,  Dong Chen ,  Paul W. Coteus ,  Alan G. Gara ,  Mark E. Giampapa ,  Philip Heidelberger ,  Burkhard D. Steinmacher-Burow ,  Todd E. Takken ,  Pavlos M. Vranas

IPC分类号： G06F15/173

CPC分类号： H05K7/20836 ,  F24F11/77 ,  G06F9/52 ,  G06F9/526 ,  G06F15/17381 ,  G06F17/142 ,  G09G5/008 ,  H04L7/0338

摘要： In a massively parallel computing system having a plurality of nodes configured in m multi-dimensions, each node including a computing device, a method for routing packets towards their destination nodes is provided which includes generating at least one of a 2m plurality of compact bit vectors containing information derived from downstream nodes. A multilevel arbitration process in which downstream information stored in the compact vectors, such as link status information and fullness of downstream buffers, is used to determine a preferred direction and virtual channel for packet transmission. Preferred direction ranges are encoded and virtual channels are selected by examining the plurality of compact bit vectors. This dynamic routing method eliminates the necessity of routing tables, thus enhancing scalability of the switch.

摘要翻译： 在具有以m多维配置的多个节点的大规模并行计算系统中，每个节点包括计算设备，提供了用于向分组朝向其目的地节点路由分组的方法，其包括生成2m个多个紧凑比特向量中的至少一个 包含从下游节点导出的信息。 存储在紧凑向量中的下行信息（诸如链路状态信息和下游缓冲器的丰满度）的多级仲裁过程被用于确定分组传输的优选方向和虚拟信道。 优选的方向范围被编码，并且通过检查多个紧凑比特向量来选择虚拟信道。 这种动态路由方法消除了路由表的必要性，从而增强了交换机的可扩展性。

公开(公告)号：US07802025B2

公开(公告)日：2010-09-21

申请号：US11768795

申请日：2007-06-26

申请人： Dong Chen ,  Alan G. Gara ,  Mark E. Giampapa ,  Philip Heidelberger ,  Burkhard Steinmacher-Burow ,  Pavlos Vranas

发明人： Dong Chen ,  Alan G. Gara ,  Mark E. Giampapa ,  Philip Heidelberger ,  Burkhard Steinmacher-Burow ,  Pavlos Vranas

IPC分类号： G06F13/28

CPC分类号： G06F15/163

摘要： A parallel computer system is constructed as a network of interconnected compute nodes to operate a global message-passing application for performing communications across the network. Each of the compute nodes includes one or more individual processors with memories which run local instances of the global message-passing application operating at each compute node to carry out local processing operations independent of processing operations carried out at other compute nodes. Each compute node also includes a DMA engine constructed to interact with the application via Injection FIFO Metadata describing multiple Injection FIFOs where each Injection FIFO may containing an arbitrary number of message descriptors in order to process messages with a fixed processing overhead irrespective of the number of message descriptors included in the Injection FIFO.

摘要翻译： 并行计算机系统被构造为互连的计算节点的网络，以操作用于在整个网络上执行通信的全局消息传递应用。 每个计算节点包括具有存储器的一个或多个单独处理器，该存储器运行在每个计算节点处操作的全局消息传递应用的本地实例，以独立于在其他计算节点执行的处理操作来执行本地处理操作。 每个计算节点还包括构造成通过描述多个注入FIFO的注入FIFO元数据与应用交互的DMA引擎，其中每个注入FIFO可以包含任意数量的消息描述符，以便处理具有固定处理开销的消息，而不管消息的数量 描述符包含在注入FIFO中。

公开(公告)号：US07486619B2

公开(公告)日：2009-02-03

申请号：US10793068

申请日：2004-03-04

申请人： Dong Chen ,  Alan G. Gara ,  Mark E. Giampapa ,  Philip Heidelberger ,  Dirk Hoenicke ,  Burkhard D. Steinmacher-Burow ,  Pavlos M. Vranas ,  Matthias Augustin Blumrich

发明人： Dong Chen ,  Alan G. Gara ,  Mark E. Giampapa ,  Philip Heidelberger ,  Dirk Hoenicke ,  Burkhard D. Steinmacher-Burow ,  Pavlos M. Vranas ,  Matthias Augustin Blumrich

IPC分类号： H04L12/28

CPC分类号： H04L49/1576 ,  H04L45/06

摘要： Multidimensional switch data networks are disclosed, such as are used by a distributed-memory parallel computer, as applied for example to computations in the field of life sciences. A distributed memory parallel computing system comprises a number of parallel compute nodes and a message passing data network connecting the compute nodes together. The data network connecting the compute nodes comprises a multidimensional switch data network of compute nodes having N dimensions, and a number/array of compute nodes Ln in each of the N dimensions. Each compute node includes an N port routing element having a port for each of the N dimensions. Each compute node of an array of Ln compute nodes in each of the N dimensions connects through a port of its routing element to an Ln port crossbar switch having Ln ports. Several embodiments are disclosed of a 4 dimensional computing system having 65,536 compute nodes.

摘要翻译： 公开了多维交换机数据网络，例如由分布式存储器并行计算机使用的，例如应用于生命科学领域的计算。 分布式存储器并行计算系统包括多个并行计算节点和将计算节点连接在一起的消息传递数据网络。 连接计算节点的数据网络包括具有N维的计算节点的多维交换机数据网络和N个维度中的每一个中的计算节点Ln的数量/数组。 每个计算节点包括具有用于N个维度中的每一个的端口的N端口路由元件。 每个N维中的Ln计算节点阵列的每个计算节点通过其路由元素的端口连接到具有Ln端口的Ln端口交叉开关。 公开了具有65,536个计算节点的四维计算系统的几个实施例。