-
公开(公告)号：US10534768B2
公开(公告)日：2020-01-14
申请号：US14981540
申请日：2015-12-28
Applicant: Amazon Technologies, Inc.
Inventor： Pradeep Jnana Madhavarapu , Raman Mittal , Benjamin Tobler , James McClellan Corey , Yan Valerie Leshinsky , Samuel James McKelvie , Anurag Windlass Gupta
Abstract: A log-structured data store may implement optimized log storage for asynchronous log updates. In some embodiments, log records may be received indicating updates to data stored for a storage client and indicating positions in a log record sequence. The log records themselves may not be guaranteed to be received according to the log record sequence. Received log records may be stored in a hot log portion of a block-based storage device according to an order in which they are received. Log records in the hot log portion may then be identified to be moved to a cold log portion of the block-based storage device in order to complete a next portion of the log record sequence. Log records may be modified, such as compressed, or coalesced, before being stored together in a data block of the cold log portion according to the log record sequence.
-
公开(公告)号：US12038906B2
公开(公告)日：2024-07-16
申请号：US18054870
申请日：2022-11-11
Applicant: Amazon Technologies, Inc.
Inventor： Anurag Windlass Gupta , Neal Fachan , Samuel James McKelvie , Laurion Darrell Burchall , Christopher Richard Newcombe , Pradeep Jnana Madhavarapu , Benjamin Tobler , James McClellan Corey
CPC classification number: G06F16/2365 , G06F11/1451 , G06F11/1471 , G06F16/23 , G06F11/2094 , G06F2201/80
Abstract: A database system may include a database service and a separate distributed storage service. The database service (or a database engine head node thereof) may be responsible for query parsing, optimization, and execution, transactionality, and consistency, while the storage service may be responsible for generating data pages from redo log records and for durability of those data pages. For example, in response to a write request directed to a particular data page, the database engine head node may generate a redo log record and send it, but not the data page, to a storage service node. The storage service node may store the redo log record and return a write acknowledgement to the database service prior to applying the redo log record. The server node may apply the redo log record and other redo log records to a previously stored version of the data page to create a current version.
-
公开(公告)号：US10609123B2
公开(公告)日：2020-03-31
申请号：US15665080
申请日：2017-07-31
Applicant: Amazon Technologies, Inc.
Inventor： Samuel James McKelvie , Anurag Windlass Gupta , Pradeep Jnana Madhavarapu , James McClellan Corey , Yan Valerie Leshinsky
Abstract: Hybrid quorum policies may be implemented for access requests to state maintained in a distributed system. An update to state maintained in a distributed system may be sent to multiple nodes in the distributed system maintain the state in order to satisfy a hybrid quorum policy for updating the state in the distributed system. The hybrid quorum policy may include individual quorum policies for multiple quorum sets including nodes of the distributed system. Acknowledgments received from nodes in the distributed system indicating that the update is performed may be evaluated according to the individual quorum policies to determine whether the hybrid quorum policy is satisfied. If the hybrid quorum policy is satisfied, then the update to the state may be considered durable.
-
公开(公告)号：US10229011B2
公开(公告)日：2019-03-12
申请号：US15413331
申请日：2017-01-23
Applicant: Amazon Technologies, Inc.
Inventor： Yan Valerie Leshinsky , James McClellan Corey , Pradeep Jnana Madhavarapu , Anurag Windlass Gupta , Benjamin Tobler , Samuel James McKelvie
Abstract: A distributed database system may implement log-structured distributed storage using a single log sequence number space. A log for a data volume may be maintained in a log-structured distributed storage system. The log may be segmented across multiple protection groups according to a partitioning of user data for the data volume. Updates to the log may be assigned a log sequence number from a log sequence number space for the data volume. A protection group may be determined for an update according to which partition of user data space the update pertains. Metadata to be included with the log record may indicate a previous log sequence number of a log record maintained at the protection group. The log record may be sent to the protection group and identified as committed based on acknowledgments received from storage nodes implementing the protection group.
-
公开(公告)号：US10223184B1
公开(公告)日：2019-03-05
申请号：US14036792
申请日：2013-09-25
Applicant: Amazon Technologies, Inc.
Inventor： Samuel James McKelvie , Benjamin Tobler , James McClellan Corey , Pradeep Jnana Madhavarapu , Oscar Ricardo Moll Thomae , Christopher Richard Newcombe , Yan Valerie Leshinsky , Anurag Windlass Gupta
Abstract: A log-structured distributed storage system may implement individual write quorums. Log records may be sent to different storage nodes of a quorum set storing data for a storage client sufficient to satisfy a write quorum requirement. For each log record, acknowledgments from storage nodes are received, and a determination is made whether the write quorum requirement is satisfied for the log record. Different log records may be maintained at different storage nodes, and still satisfy the write quorum requirement such that in some embodiments no one storage node may maintain all of the log records sent to storage nodes in the quorum set.
-
Patent Agency Ranking
6.发明授权Self-describing data blocks of a minimum atomic write size for a data store 有权用于数据存储的最小原子写入大小的自描述数据块公开(公告)号：US09047189B1
公开(公告)日：2015-06-02
申请号：US13903674
申请日：2013-05-28
Applicant: Amazon Technologies, Inc.
Inventor： Anurag Windlass Gupta , James McClellan Corey , Yan Valerie Leshinsky , Pradeep Jnana Madhavarapu , Samuel James McKelvie
CPC classification number: G06F11/0793 , G06F3/0619 , G06F3/064 , G06F3/067 , G06F3/0689 , G06F11/0709 , G06F11/0751 , G06F11/0787 , G06F11/079 , G06F11/10 , G06F11/1004 , G06F11/1076
Abstract: Self-describing data blocks of a minimum atomic write size may be stored for a data store. Data may be received for storage in a data block of a plurality of data blocks at a persistent storage device that are equivalent to a minimum atomic write size for the persistent storage device. Metadata may be generated for the data that includes an error detection code which is generated for the data and the metadata together. The data and the metadata are sent to the persistent storage device to store together in the data block. An individual atomic write operation may write together the data and the metadata in the data block. When accessed, the error detection code is applicable to detect errors. The metadata may also be applicable to determine whether the data is stored for a currently assigned purpose or a previously assigned purpose of the data block.
Abstract translation: 可以为数据存储器存储最小原子写入大小的自描述数据块。 可以接收数据以存储在永久存储设备处的多个数据块的数据块中，其等同于持久存储设备的最小原子写入大小。 可以为包括为数据和元数据生成的错误检测码的数据生成元数据。 将数据和元数据发送到持久存储设备，以一起存储在数据块中。 单独的原子写入操作可以将数据和元数据一起写入数据块中。 访问时，错误检测码适用于检测错误。 元数据还可以适用于确定数据是否存储在当前分配的目的或数据块的先前分配的目的上。
-
公开(公告)号：US10437721B2
公开(公告)日：2019-10-08
申请号：US15284414
申请日：2016-10-03
Applicant: Amazon Technologies, Inc.
Inventor： Yan Valerie Leshinsky , James McClellan Corey , Samuel James McKelvie , Oscar Ricardo Moll Thomae , Pradeep Jnana Madhavarapu
Abstract: A log-structured data store may implement efficient garbage collection. Log records may be maintained in data blocks according to a log record sequence. Based, at least in part, on a log reclamation point, the log records may be evaluated to identify data blocks to reclaim that have log records in the log sequence prior to the log reclamation point. New versions of data pages updated by log records in the identified data blocks may be generated and stored in base page storage for the log structured data store. The identified data blocks may then be reclaimed for storing new data.
-
公开(公告)号：US09785510B1
公开(公告)日：2017-10-10
申请号：US14274366
申请日：2014-05-09
Applicant: Amazon Technologies, Inc.
Inventor： Pradeep Jnana Madhavarapu , Anurag Windlass Gupta , Yan Valerie Leshinsky , Samuel James McKelvie , James McClellan Corey
CPC classification number: G06F11/1451
Abstract: A log-structured data store implementing data backup may implement variable data replication. Write requests may be received at different storage nodes maintaining respective replicas of a portion of a log for data maintained in the log-structured data store. Log records indicating the write requests may be stored in the respective replicas of the log portions at the different storage nodes. The log records may be sent to a backup data store to be durability persisted as part of an archived version of the log. At some of the storage nodes, in response to determining that the log records have been durably persisted in the backup data store, storage space for the log records may be reclaimed. In other remaining storage nodes, the log records may be retained and made accessible for servicing read requests.
-
公开(公告)号：US20170132091A1
公开(公告)日：2017-05-11
申请号：US15413331
申请日：2017-01-23
Applicant: Amazon Technologies, Inc.
Inventor： Yan Valerie Leshinsky , James McClellan Corey , Pradeep Jnana Madhavarapu , Anurag Windlass Gupta , Benjamin Tobler , Samuel James McKelvie
CPC classification number: G06F11/1471 , G06F3/0619 , G06F3/064 , G06F3/067 , G06F11/00 , G06F11/0709 , G06F11/0727 , G06F2201/805 , G06F2201/82
Abstract: A distributed database system may implement log-structured distributed storage using a single log sequence number space. A log for a data volume may be maintained in a log-structured distributed storage system. The log may be segmented across multiple protection groups according to a partitioning of user data for the data volume. Updates to the log may be assigned a log sequence number from a log sequence number space for the data volume. A protection group may be determined for an update according to which partition of user data space the update pertains. Metadata to be included with the log record may indicate a previous log sequence number of a log record maintained at the protection group. The log record may be sent to the protection group and identified as committed based on acknowledgments received from storage nodes implementing the protection group.
-
公开(公告)号：US09465693B2
公开(公告)日：2016-10-11
申请号：US14727644
申请日：2015-06-01
Applicant: Amazon Technologies, Inc.
Inventor： Anurag Windlass Gupta , James McClellan Corey , Yan Valerie Leshinsky , Pradeep Jnana Madhavarapu , Samuel James McKelvie
CPC classification number: G06F11/0793 , G06F3/0619 , G06F3/064 , G06F3/067 , G06F3/0689 , G06F11/0709 , G06F11/0751 , G06F11/0787 , G06F11/079 , G06F11/10 , G06F11/1004 , G06F11/1076
Abstract: Self-describing data blocks of a minimum atomic write size may be stored for a data store. Data may be received for storage in a data block of a plurality of data blocks at a persistent storage device that are equivalent to a minimum atomic write size for the persistent storage device. Metadata may be generated for the data that includes an error detection code which is generated for the data and the metadata together. The data and the metadata are sent to the persistent storage device to store together in the data block. An individual atomic write operation may write together the data and the metadata in the data block. When accessed, the error detection code is applicable to detect errors. The metadata may also be applicable to determine whether the data is stored for a currently assigned purpose or a previously assigned purpose of the data block.
-
-
-
-
-
-
-
-
-
Search Results
28
records
(took 0.027 seconds)
