Abstract:
A method, system, and computer-readable storage device for transferring data from a database management system to a data analytics system is described herein. In one aspect, a database management node storing a data partition of a database table that is distributed across a plurality of database management nodes of a database management system receives a data transfer request from a data analytics node of a data analytics node cluster. The data transfer request may include a data transfer policy for splitting the data partition onto the data analytics node cluster. A data split may be identified from the data partition. The data split may be a subset of the data partition. The data split is communicated to the data analytics node according to the data transfer policy.
Abstract:
A technique includes distributing a plurality of tasks among a plurality of worker nodes to perform a processing operation on an array. Each task is associated with a set of a least one data block of the array, and an order of the tasks is defined by an array-based programming language. Distribution of the tasks includes, for at least one of the worker nodes, selectively reordering the order defined by the array-based programming language to regulate an amount of data transferred to the worker node.
Abstract:
A memory has a shared data object containing shared data for a plurality of program instances. An allocation routine allocates a respective memory region corresponding to the shared data object to each of the plurality of program instances, where each of the memory regions contains a header part and a data part, where the data part corresponds to the shared data and the header part contains information relating to the data part, and the header part is private to the corresponding program instance. The allocation routine maps the shared data to the memory regions using a mapping technique that avoids copying the shared data to each of the data parts as part of allocating the corresponding memory region.
Abstract:
Log analysis can include transferring compiled log analysis code, executing log analysis code, and performing a log analysis on the executed log analysis code.
Abstract:
A technique includes receiving a user input in an array-oriented database. The user input indicates a database operation and processing a plurality of chunks of data stored by the database to perform the operation. The processing in dudes selectively distributing the processing of the plurality of chunks between a first group of at least one central processing unit and a second group of at least one co-processor.
Abstract:
A technique includes distributing a plurality of tasks among a plurality of worker nodes to perform a processing operation on an array. Each task is associated with a set of a least one data block of the array, and an order of the tasks is defined by an array-based programming language. Distribution of the tasks includes, for at least one of the worker nodes, selectively reordering the order defined by the array-based programming language to regulate an amount of data transferred to the worker node.
Abstract:
A method, system, and computer-readable storage device for transferring data from a database management system to a data analytics system is described herein. In one aspect, a database management node storing a data partition of a database table that is distributed across a plurality of database management nodes of a database management system receives a data transfer request from a data analytics node of a data analytics node cluster. The data transfer request may include a data transfer policy for splitting the data partition onto the data analytics node cluster. A data split may be identified from the data partition. The data split may be a subset of the data partition. The data split is communicated to the data analytics node according to the data transfer policy.
Abstract:
A sparse array is partitioned into first partitions and a second array is partitioned into second partitions based on an invariant relationship between the sparse array and the second array. The sparse array and the second array are associated with a computation involving the sparse array and the second array. The first partitions and the corresponding second partitions are distributed to workers. A different first partition and corresponding second partition is distributed to each of the workers. Third partitions of the sparse array and corresponding fourth partitions of the second array are determined based on the invariant relationship and measurements of load are received from each of the workers. At least one of the first partitions and the corresponding second partition is different from one of the third partitions and the corresponding fourth partition. The at least one of the first partitions and the corresponding second partition that is different is redistributed among the workers. A different third partition and corresponding fourth partition is executed by each of the workers.