Lottery-based resource allocation with capacity guarantees

    公开(公告)号:US11029999B1

    公开(公告)日:2021-06-08

    申请号:US16124118

    申请日:2018-09-06

    Abstract: Methods, systems, and computer-readable media for lottery-based resource allocation with capacity guarantees are disclosed. A job request is received from a first client. The job request is submitted to a capacity management system that schedules jobs in a pool of compute resources. The pool comprises a first quantity of one or more slots and a second quantity of one or more slots. The second quantity is associated with a capacity guarantee for a second client. It is determined that the first quantity of one or more slots are in use by one or more jobs initiated prior to receiving the job request. It is determined that the second quantity of one or more slots comprises an available slot. The available slot is allocated to the job request.

    Data store indexing engine with automated refresh

    公开(公告)号:US11934370B1

    公开(公告)日:2024-03-19

    申请号:US15838299

    申请日:2017-12-11

    CPC classification number: G06F16/2272 G06F16/2291 G06F16/24 G06F16/81

    Abstract: Systems and methods are disclosed to implement an indexing engine that maintains an index in an index store for a storage object in a data store. In embodiments, the index store may be implemented using an in-memory storage cluster separate from the data store. The storage object may have multiple indexes, which may have different filtering or sorting criteria for the data. In embodiments, updates to the storage object are received as an update stream by the indexing engine. Based on configurable indexing rules, the indexing engine applies the updates to the appropriate indexes. To service a query to the data store, a query engine first retrieves a set of keys satisfying the query from the index store, and then data corresponding to the keys from the data store or another index. In embodiments, the index may be refreshed via touch updates of selected data in the storage object.

    Non-native transactional support for distributed computing environments

    公开(公告)号:US11314700B1

    公开(公告)日:2022-04-26

    申请号:US16055904

    申请日:2018-08-06

    Abstract: Techniques are generally described for adding transactional support to a distributed storage environment. In various examples, first data may be written to a first set of locations in a distributed computer-readable non-transitory storage system through a non-transactional file system interface. In various further examples, metadata associated with the first data may be generated during the writing of the first data. In some examples, the metadata may be stored associated with the first data in at least a second location in a second computer-readable non-transitory memory. In some examples, a manifest may be generated defining a transactional commit of at least a portion of the first data. In some examples, the manifest may be generated by processing the metadata using first committer logic. In some further examples, the manifest may be stored in a third computer-readable non-transitory memory.

    Indexing partitions using distributed bloom filters

    公开(公告)号:US11531666B1

    公开(公告)日:2022-12-20

    申请号:US16998922

    申请日:2020-08-20

    Abstract: Methods, systems, and computer-readable media for indexing partitions using distributed Bloom filters are disclosed. A data indexing system generates a plurality of indices for a plurality of partitions in a distributed object store. The indices comprise a plurality of Bloom filters. An individual one of the Bloom filters corresponds to one or more fields of an individual one of the partitions. Using the Bloom filters, the data indexing system determines a first portion of the partitions that possibly comprise a value and a second portion of the partitions that do not comprise the value. Based (at least in part) on a scan of the first portion of the partitions and not the second portion of the partitions, the data indexing system determines one or more partitions of the first portion of the partitions that comprise the value.

    Transformation specification format for multiple execution engines

    公开(公告)号:US11347548B2

    公开(公告)日:2022-05-31

    申请号:US16848715

    申请日:2020-04-14

    Abstract: Methods, systems, and computer-readable media for a transformation specification format for multiple execution engines are disclosed. A transformation specification is expressed according to a transformation specification format. The transformation specification represents a polytree or graph linking one or more data producer nodes, one or more data transformation nodes, and one or more data consumer nodes. An execution engine is selected from among a plurality of available execution engines for execution of the transformation specification. The execution engine is used to acquire data from one or more data producers corresponding to the one or more data producer nodes, perform one or more transformations of the data corresponding to the one or more data transformation nodes, and output one or more results of the one or more transformations to one or more data consumers corresponding to the one or more data consumer nodes.

    TRANSFORMATION SPECIFICATION FORMAT FOR MULTIPLE EXECUTION ENGINES

    公开(公告)号:US20200241920A1

    公开(公告)日:2020-07-30

    申请号:US16848715

    申请日:2020-04-14

    Abstract: Methods, systems, and computer-readable media for a transformation specification format for multiple execution engines are disclosed. A transformation specification is expressed according to a transformation specification format. The transformation specification represents a polytree or graph linking one or more data producer nodes, one or more data transformation nodes, and one or more data consumer nodes. An execution engine is selected from among a plurality of available execution engines for execution of the transformation specification. The execution engine is used to acquire data from one or more data producers corresponding to the one or more data producer nodes, perform one or more transformations of the data corresponding to the one or more data transformation nodes, and output one or more results of the one or more transformations to one or more data consumers corresponding to the one or more data consumer nodes.

Patent Agency Ranking