PARALLEL DATA STORE SYSTEM, COMPUTER-READABLE RECORDING MEDIUM STORING PROGRAM, AND PARALLEL DATA STORE METHOD

    公开(公告)号:US20230252003A1

    公开(公告)日:2023-08-10

    申请号:US17988354

    申请日:2022-11-16

    Inventor: HIROKI OHTSUJI

    CPC classification number: G06F16/1858 G06F11/3419

    Abstract: A management server in a parallel data system stores a correspondence-relationship between a first response-time for communication-processing and a second response-time for data-processing, executed by each of a plurality of data servers in relation to first processing by a client node, acquires, at a time of execution of second processing by the client node, a third response-time desired for communication-processing and a fourth response-time desired for data-processing, which are related to second processing, in each of a plurality of data servers, based on the first response-time and the second response-time, determines combinations of the data servers used to execute the second processing, based on the third response-time and fourth response-time, and selects a combination that satisfies a response-time to be satisfied by the second processing and that includes a smallest number of processor cores allocated to the communication-processing of a data server, among the determined combinations.

    Hybrid model of fine-grained locking and data partitioning

    公开(公告)号:US12013818B2

    公开(公告)日:2024-06-18

    申请号:US17717294

    申请日:2022-04-11

    Applicant: NetApp Inc.

    CPC classification number: G06F16/1774 G06F16/1858

    Abstract: Presented herein are methods, non-transitory computer readable media, and devices for integrating a hybrid model of fine-grained locking and data-partitioning wherein fine-grained locking is added to existing systems that are based on hierarchical data-partitioning in order in increase parallelism with minimal code re-write. Methods for integrating a hybrid model of fine-grained locking and data-partitioning are disclosed which include: creating, by a network storage server, a plurality of domains for execution of processes of the network storage server, the plurality of domains including a domain; creating a hierarchy of storage filesystem subdomains within the domain, wherein each of the subdomains corresponds to one or more types of processes, wherein at least one of the storage filesystem subdomains maps to a data object that is locked via fine-grained locking; and assigning processes for simultaneous execution by the storage filesystem subdomains within the domain and the at least one subdomain that maps to the data object locked via fine-grained locking.

    COMPUTER WORKLOAD MANAGER
    6.
    发明申请
    COMPUTER WORKLOAD MANAGER 审中-公开
    计算机工作负载管理器

    公开(公告)号:US20160026553A1

    公开(公告)日:2016-01-28

    申请号:US14337668

    申请日:2014-07-22

    Applicant: Cray Inc.

    Inventor: Peter Piela

    Abstract: A computer-implemented method includes: scheduling computing jobs; processing data by executing the computing jobs; arranging the data in a file system; managing the arranging the data by monitoring a performance parameter of the file system and extracting information about the scheduling, and tuning one of the arranging and the scheduling based on the performance parameter and the information about the scheduling. An article of manufacture includes a computer-readable medium storing signals representing instructions for a computer program executing the method.

    Abstract translation: 计算机实现的方法包括:调度计算任务; 通过执行计算作业处理数据; 将数据排列在文件系统中; 通过监视文件系统的性能参数并提取关于调度的信息来管理数据的排列,并且基于性能参数和关于调度的信息调整排列和调度中的一个。 一种制品包括存储表示执行该方法的计算机程序的指令的信号的计算机可读介质。

    USING NETWORK ADDRESSABLE NON-VOLATILE MEMORY FOR HIGH-PERFORMANCE NODE-LOCAL INPUT/OUTPUT
    7.
    发明申请
    USING NETWORK ADDRESSABLE NON-VOLATILE MEMORY FOR HIGH-PERFORMANCE NODE-LOCAL INPUT/OUTPUT 审中-公开
    使用网络可寻址的非易失性存储器进行高性能节点本地输入/输出

    公开(公告)号:US20140337457A1

    公开(公告)日:2014-11-13

    申请号:US14274395

    申请日:2014-05-09

    Abstract: Data storage systems and methods for storing data in computing nodes of a super computer or compute cluster are described herein. The super computer storage may be integrated with or coupled with a primary storage system. In addition to a CPU and memory, non-volatile memory is included with the computing nodes as local storage. A high speed interconnect remote direct memory access (HRI) unit is also included with each computing node. When data bursts occur, data may be stored by a first computing node on the local storage of a second computing node through the HRI units of the computing nodes, bypassing the CPU of the second computing node. Further, the local storage of other computing nodes may be used to store redundant copies of data from a first computing node to make the super computer data resilient while not interfering with the CPU of the other computing nodes.

    Abstract translation: 这里描述了用于在超级计算机或计算群集的计算节点中存储数据的数据存储系统和方法。 超级计算机存储器可以与主存储系统集成或耦合。 除了CPU和内存之外,非易失性存储器作为本地存储器包含在计算节点中。 每个计算节点还包括高速互连远程直接存储器访问(HRI)单元。 当数据突发发生时,数据可以通过计算节点的HRI单元绕过第二计算节点的CPU,由第一计算节点存储在第二计算节点的本地存储器上。 此外,其他计算节点的本地存储可以用于存储来自第一计算节点的数据的冗余副本,以使得超级计算机数据具有弹性,而不干扰其他计算节点的CPU。

    Parallel Filesystem Traversal For Transparent Mirroring of Directories and Files
    8.
    发明申请
    Parallel Filesystem Traversal For Transparent Mirroring of Directories and Files 审中-公开
    并行文件系统遍历目录和文件的透明镜像

    公开(公告)号:US20140330781A1

    公开(公告)日:2014-11-06

    申请号:US14335932

    申请日:2014-07-20

    Abstract: A system and method for parallel file system traversal using multiple job executors is disclosed. The system includes a pool of job executors, a job queue, and a trigger tracker. An object, representative of a node in the filesystem, is added (i.e., pushed) to the job queue for processing by an job executor. The job queue assigns (i.e., pops) objects to job executors in accordance to a LIFO (Last In First Out) ordering. Then the job executor performs an action such as copy. In one embodiment, the trigger tracker follows the processing of a child nodes to a particular child node. Thus, the filesystem is being traversed by several job executors at the same time.

    Abstract translation: 公开了一种使用多个作业执行器并行文件系统遍历的系统和方法。 该系统包括一组作业执行器,一个作业队列和一个触发器跟踪器。 代表文件系统中的节点的对象被添加(即被推送)到作业队列以供作业执行器处理。 作业队列根据LIFO(先进先出)排序将(即弹出)对象分配给作业执行器。 然后,作业执行器执行诸如复制的操作。 在一个实施例中,触发跟踪器遵循对特定子节点的子节点的处理。 因此,文件系统同时被多个作业执行器遍历。

    Systems for analyzing and computing data items
    9.
    发明申请
    Systems for analyzing and computing data items 审中-公开
    用于分析和计算数据项的系统

    公开(公告)号:US20020083424A1

    公开(公告)日:2002-06-27

    申请号:US09989098

    申请日:2001-11-20

    Abstract: A computer system splits a data space to partition data between processors or processes. The data space may be split into sub-regions which need not be orthogonal to the axes defined by the data space's parameters, using a decision tree. The decision tree can have neural networks in each of its non-terminal nodes that are trained on, and are used to partition, training data. Each terminal, or leaf, node can have a hidden layer neural network trained on the training data that reaches the terminal node. The training of the non-terminal nodes' neural networks can be performed on one processor and the training of the leaf nodes' neural networks can be run on separate processors. Different target values can be used for the training of the networks of different non-terminal nodes. The non-terminal node networks may be hidden layer neural networks. Each non-terminal node automatically may send a desired ratio of the training records it receives to each of its child nodes, so the leaf node networks each receives approximately the same number of training records. The system may automatically configures the tree to have a number of leaf nodes equal to the number of separate processors available to train leaf node networks. After the non-terminal and leaf node networks have been trained, the records of a large data base can be passed through the tree for classification or for estimation of certain parameter values.

    Abstract translation: 计算机系统将数据空间拆分为处理器或进程之间的数据分区。 可以使用决策树将数据空间拆分成不需要与由数据空间参数定义的轴正交的子区域。 决策树可以在其非终端节点中的每个训练数据上进行训练并用于分割训练数据的神经网络。 每个终端或叶节点可以具有对到达终端节点的训练数据训练的隐层神经网络。 可以在一个处理器上执行非终端节点神经网络的训练,并且可以在单独的处理器上运行叶节点的神经网络的训练。 不同目标值可用于不同非终端节点网络的训练。 非终端节点网络可以是隐层神经网络。 每个非终端节点可自动发送其接收到的每个子节点的培训记录的期望比例,因此叶节点网络每个接收大约相同数量的训练记录。 该系统可以自动地配置该树以使得多个叶节点等于可用于训练叶节点网络的单独处理器的数量。 在非终端和叶节点网络被训练之后,大数据库的记录可以通过树进行分类或估计某些参数值。

Patent Agency Ranking