Automated provisioning for database performance

    公开(公告)号:US11782926B2

    公开(公告)日:2023-10-10

    申请号:US17573897

    申请日:2022-01-12

    CPC classification number: G06F16/24545 G06F16/217 G06N20/00 G06N20/20

    Abstract: Embodiments utilize trained query performance machine learning (QP-ML) models to predict an optimal compute node cluster size for a given in-memory workload. The QP-ML models include models that predict query task runtimes at various compute node cardinalities, and models that predict network communication time between nodes of the cluster. Embodiments also utilize an analytical model to predict overlap between predicted task runtimes and predicted network communication times. Based on this data, an optimal cluster size is selected for the workload. Embodiments further utilize trained data capacity machine learning (DC-ML) models to predict a minimum number of compute nodes needed to run a workload. The DC-ML models include models that predict the size of the workload dataset in a target data encoding, models that predict the amount of memory needed to run the queries in the workload, and models that predict the memory needed to accommodate changes to the dataset.

    Disk drive failure prediction with neural networks

    公开(公告)号:US11579951B2

    公开(公告)日:2023-02-14

    申请号:US16144912

    申请日:2018-09-27

    Abstract: Techniques are described herein for predicting disk drive failure using a machine learning model. The framework involves receiving disk drive sensor attributes as training data, preprocessing the training data to select a set of enhanced feature sequences, and using the enhanced feature sequences to train a machine learning model to predict disk drive failures from disk drive sensor monitoring data. Prior to the training phase, the RNN LSTM model is tuned using a set of predefined hyper-parameters. The preprocessing, which is performed during the training and evaluation phase as well as later during the prediction phase, involves using predefined values for a set of parameters to generate the set of enhanced sequences from raw sensor reading. The enhanced feature sequences are generated to maintain a desired healthy/failed disk ratio, and only use samples leading up to a last-valid-time sample in order to honor a pre-specified heads-up-period alert requirement.

    AUTOMATED PROVISIONING FOR DATABASE PERFORMANCE

    公开(公告)号:US20220138199A1

    公开(公告)日:2022-05-05

    申请号:US17573897

    申请日:2022-01-12

    Abstract: Embodiments utilize trained query performance machine learning (QP-ML) models to predict an optimal compute node cluster size for a given in-memory workload. The QP-ML models include models that predict query task runtimes at various compute node cardinalities, and models that predict network communication time between nodes of the cluster. Embodiments also utilize an analytical model to predict overlap between predicted task runtimes and predicted network communication times. Based on this data, an optimal cluster size is selected for the workload. Embodiments further utilize trained data capacity machine learning (DC-ML) models to predict a minimum number of compute nodes needed to run a workload. The DC-ML models include models that predict the size of the workload dataset in a target data encoding, models that predict the amount of memory needed to run the queries in the workload, and models that predict the memory needed to accommodate changes to the dataset.

    GRADIENT-BASED AUTO-TUNING FOR MACHINE LEARNING AND DEEP LEARNING MODELS

    公开(公告)号:US20220027746A1

    公开(公告)日:2022-01-27

    申请号:US17499945

    申请日:2021-10-13

    Abstract: Herein, horizontally scalable techniques efficiently configure machine learning algorithms for optimal accuracy and without informed inputs. In an embodiment, for each particular hyperparameter, and for each epoch, a computer processes the particular hyperparameter. An epoch explores one hyperparameter based on hyperparameter tuples. A respective score is calculated from each tuple. The tuple contains a distinct combination of values, each of which is contained in a value range of a distinct hyperparameter. All values of a tuple that belong to the particular hyperparameter are distinct. All values of a tuple that belong to other hyperparameters are held constant. The value range of the particular hyperparameter is narrowed based on an intersection point of a first line based on the scores and a second line based on the scores. A machine learning algorithm is optimally configured from repeatedly narrowed value ranges of hyperparameters. The configured algorithm is invoked to obtain a result.

    Tail-based top-N query evaluation
    85.
    发明授权

    公开(公告)号:US11194801B2

    公开(公告)日:2021-12-07

    申请号:US16446636

    申请日:2019-06-20

    Abstract: Techniques are described for executing a query with a top-N clause to select a first N-number of rows in a data source arranged at least according to a first key and a second key of the data source using a first sort order respectively specified for the first key and a second sort order respectively specified for the second key by the query. The data source may include one or more tiles that include at least a portion of the first key and the second key. To execute the query, in an embodiment, a DBMS determines, in a first vector of first key values that are in a first tile, row identifiers identifying entries of the first vector that contain values equal to a tail value that follows a particular top number of the first key values. The DBMS may select, from a second vector of values of the second key in the first tile, second key values identified based on the determined row identifiers of the first vector. In an embodiment, the DBMS generates a result set of the query that includes at least a value from the second key values selected from the second vector based on the determined first row identifiers.

    Adaptive resolution histogram on complex datatypes

    公开(公告)号:US11048679B2

    公开(公告)日:2021-06-29

    申请号:US15798959

    申请日:2017-10-31

    Abstract: Techniques herein map between key spaces to generate a balanced adaptive resolution histogram for dataset partitioning. In embodiments, a computer (C) creates a mapping that associates sparse keys (SKs) with distinct dense keys. C constructs a trie by processing each item of a dataset as follows. Based on the item, C obtains an SK. C navigates from a root NT (node of the trie) to a particular NT based on a sequence of dense digits (SDD). Each dense digit of the SDD is based on the mapping. Each NT identifies a dense prefix comprising dense digits. C assigns the item to a target node based on a threshold and count of items assigned to a subtree rooted at the particular node. C determines a range of SKs for each partition of the dataset, based on: an item count for a node or subtree, dense prefixes of NTs, and the mapping.

    Distributed relational dictionaries

    公开(公告)号:US10810195B2

    公开(公告)日:2020-10-20

    申请号:US15861212

    申请日:2018-01-03

    Abstract: Techniques related to distributed relational dictionaries are disclosed. In some embodiments, one or more non-transitory storage media store a sequence of instructions which, when executed by one or more computing devices, cause performance of a method. The method involves generating, by a query optimizer at a distributed database system (DDS), a query execution plan (QEP) for generating a code dictionary and a column of encoded database data. The QEP specifies a sequence of operations for generating the code dictionary. The code dictionary is a database table. The method further involves receiving, at the DDS, a column of unencoded database data from a data source that is external to the DDS. The DDS generates the code dictionary according to the QEP. Furthermore, based on joining the column of unencoded database data with the code dictionary, the DDS generates the column of encoded database data according to the QEP.

    Efficient partitioning of relational data

    公开(公告)号:US10592531B2

    公开(公告)日:2020-03-17

    申请号:US15438521

    申请日:2017-02-21

    Abstract: Techniques for non-power-of-two partitioning of a data set as well as generation and selection of partition schemes for the data set. In an embodiment, one or more iterations of a partition scheme is for a non-power-of-two number of partitions. Extended hash partitioning may be used to partition a data set into a non-power-of-two number of partitions by determining the partition identifier of each tuple of the data set using the extended hash partitioning algorithm. In an embodiment, multiple partition schemes are generated for multiple data sets, based on properties of the data sets and/or availability of computing resources for the partition operation or the subsequent operation to the partition operation. The generated partition schemes may use non-power-of-two partitioning for one or more iterations of a generated partition scheme. The most optimal partition scheme may be selected from the generated partition schemes based on optimization policies.

    Scalable distributed computation framework for data-intensive computer vision workloads

    公开(公告)号:US10469822B2

    公开(公告)日:2019-11-05

    申请号:US15471710

    申请日:2017-03-28

    Abstract: Techniques described herein provide methods and systems for scalable distribution of computer vision workloads. In an embodiment, a method comprises receiving, at each of a first node and a second node of a distributed system of nodes, two images. The first image comprises a first set of pixels and the second image comprising a second set of pixels. The method further comprises shifting, at the first node, each pixel of the first set of pixels of the first image in a uniform direction by a first number of pixels to form a first shifted image and shifting, at the second node, each pixel of the first set of pixels of the first image in the uniform direction by a second number of pixels to form a second shifted image. The second number of pixels is different from the first number of pixels. The method further comprises overlaying each of the first shifted image and the second shifted image with the second image, such that each pixel of the first shifted image and second shifted image has a corresponding pixel in the second image. The method further comprises creating, at the first node, a first disparity map that indicates, for each pixel of the first shifted image, a level of similarity between the pixel of the first shifted image and the corresponding pixel in the second image and creating, at the second node, a second disparity map that indicates, for each pixel of the second shifted image, a level of similarity between the pixel of the second shifted image and the corresponding pixel in the second image.

    Application-level dynamic scheduling of network communication for efficient re-partitioning of skewed data

    公开(公告)号:US10263893B2

    公开(公告)日:2019-04-16

    申请号:US15372224

    申请日:2016-12-07

    Abstract: Techniques are provided for using decentralized lock synchronization to increase network throughput. In an embodiment, a first computer sends, to a second computer comprising a lock, a request to acquire the lock. In response to receiving the lock acquisition request, the second computer detects whether the lock is available. If the lock is unavailable, then the second computer replies by sending a denial to the first computer. Otherwise, the second computer sends an exclusive grant of the lock to the first computer. While the first computer has acquired the lock, the first computer sends data to the second computer. Afterwards, the first computer sends a request to release the lock to the second computer. This completes one duty cycle of the lock, and the lock is again available for acquisition.

Patent Agency Ranking