Managing partitions in a scalable environment

    公开(公告)号:US11561841B2

    公开(公告)日:2023-01-24

    申请号:US16945612

    申请日:2020-07-31

    Abstract: Systems and methods are provided that enable a general framework for partitioning application-defined jobs in a scalable environment. The general framework decouples partitioning of a job from the other aspects of the job. As a result, the effort required to define the application-defined job is reduced or minimized, as the user is not required to provide a partitioning algorithm. The general framework also facilitates management of masters and servers performing computations within the distributed environment.

    Hybrid garbage collection in a distributed storage system

    公开(公告)号:US10789223B2

    公开(公告)日:2020-09-29

    申请号:US15080474

    申请日:2016-03-24

    Abstract: In various embodiments, methods and systems for implementing garbage collection in distributed storage systems are provided. The distributed storage system operates based on independent management of metadata of extent and stream data storage resources. A hybrid garbage collection system based on reference counting garbage collection operations and mark-and-sweep garbage collection operations is implemented. An extent lifetime table that tracks reference weights and mark sequences for extents is initialized and updated based on indications from extent managers and stream managers, respectively. Upon determining that an extent is to be handed-off from weighted reference counting garbage collection operations to mark-and-sweep garbage collection operations, a reference weight field for the extent is voided and a mark sequence field of the extent is updated. The mark sequence field is updated with a latest global sequence number. The mark-and-sweep garbage collection operations are utilized to reclaim the extent when the extent is no longer referenced.

    ASYNCHRONOUS REPLICATION IN A DISTRIBUTED STORAGE ENVIRONMENT
    6.
    发明申请
    ASYNCHRONOUS REPLICATION IN A DISTRIBUTED STORAGE ENVIRONMENT 审中-公开
    分布式存储环境中的异常重现

    公开(公告)号:US20160140202A1

    公开(公告)日:2016-05-19

    申请号:US15004620

    申请日:2016-01-22

    Abstract: Embodiments of the present invention relate to asynchronously replicating data in a distributed computing environment. To achieve asynchronous replication, data received at a primary data store may be annotated with information, such as an identifier of the data. The annotated data may then be communicated to a secondary data store, which may then write the data and annotated information to one or more logs for eventual replay and committal at the secondary data store. The primary data store may communicate an acknowledgment of success in committing the data at the primary data store as well as of success in writing the data to the secondary data store. Additional embodiments may include committing the data at the secondary data store in response to receiving an instruction that authorizes committal of data through an identifier.

    Abstract translation: 本发明的实施例涉及在分布式计算环境中异步复制数据。 为了实现异步复制,在主数据存储处接收到的数据可以用诸如数据的标识符的信息来注释。 然后可以将注释的数据传送到辅助数据存储器,然后辅助数据存储器可以将数据和注释的信息写入一个或多个日志以在次要数据存储器上最终重放和提交。 主数据存储器可以在主数据存储器上传送提交数据的成功确认以及将数据写入辅助数据存储器的成功。 另外的实施例可以包括在辅助数据存储器处响应于接收授权通过标识符提交数据的指令来提交数据。

Patent Agency Ranking