-
公开(公告)号:US09633051B1
公开(公告)日:2017-04-25
申请号:US14032883
申请日:2013-09-20
Applicant: Amazon Technologies, Inc.
Inventor: Maximiliano Maccanti , Timothy Andrew Rath , Rama Krishna Sandeep Pokkunuri , Akshat Vig , Clarence Wing Yin Ng , Srivaths Badrinath Copparam , Rajaprabhu Thiruchi Loganathan , Wei Xiao , William Alexander Stevenson
CPC classification number: G06F11/1469 , G06F11/1451 , G06F11/1458 , G06F11/2094 , G06F2201/80
Abstract: A system that implements a data storage service may store data for a database table in multiple replicated partitions on respective storage nodes. In response to a request to back up a table, the service may back up individual partitions of the table to a remote storage system independently and (in some cases) in parallel, and may update (or create) and store metadata about the table and its partitions on storage nodes of the data storage service and/or in the remote storage system. Backing up each partition may include exporting it from the database in which the table is stored, packaging and compressing the exported partition for upload, and uploading the exported, packaged, and compressed partition to the remote storage system. The remote storage system may be a key-value durable storage system in which each backed-up partition is accessible using its partition identifier as the key.
-
公开(公告)号:US11036591B2
公开(公告)日:2021-06-15
申请号:US16035465
申请日:2018-07-13
Applicant: Amazon Technologies, Inc.
Inventor: Maximiliano Maccanti , Timothy Andrew Rath , Rama Krishna Sandeep Pokkunuri , Akshat Vig , Clarence Wing Yin Ng , Srivaths Badrinath Copparam , Rajaprabhu Thiruchi Loganathan , Wei Xiao , William Alexander Stevenson
Abstract: A system that implements a data storage service may store data for database tables in multiple replicated partitions on respective storage nodes. In response to a request to restore a given table that was backed up in a remote storage system (e.g., key-value durable storage system), the service may create a new table, and may import a copy of each of the partitions of the given table from the remote storage system into the new table. The request to restore the table may specify a modified value for a configuration parameter for the table or for one of its partitions. The service may apply the new configuration parameter value to the table or its partitions during the restore operation. The new configuration parameter value may indicate an increase or decrease in storage capacity or throughput capacity, and its application may automatically trigger a partition split or move operation.
-
公开(公告)号:US09489443B1
公开(公告)日:2016-11-08
申请号:US13902419
申请日:2013-05-24
Applicant: Amazon Technologies, Inc.
Inventor: Kiran-Kumar Muniswamy-Reddy , Wei Xiao , Stefano Stefani , Xianglong Huang , Clarence Wing Yin Ng , Jai Vasanth
IPC: G06F17/30
CPC classification number: G06F17/30584 , G06F17/30575
Abstract: A system that implements a data storage service may store data in multiple replicated partitions on respective computing nodes on behalf of clients. A storage node may, based on the amount of provisioned resources on a given storage device or logical volume, identify candidate partition management operations to be performed, and may send information about the operations to a central partition management scheduler. The scheduler may apply a global prioritization scheme to determine an order in which to perform the candidate operations. The order may be based on whether the operations include partition splits or partition moves, whether they aim to reduce provisioned storage capacity or reduce throughput capacity on a storage device or logical volume, whether they conflict with each other, whether the total number of partitions (or replicas thereof) involved in partition management at any given time exceeds a pre-determined limit, or whether they were requested by clients.
Abstract translation: 实现数据存储服务的系统可以代表客户端将数据存储在各个计算节点上的多个复制分区中。 存储节点可以基于给定存储设备或逻辑卷上的供应资源量来识别要执行的候选分区管理操作,并且可以向中央分区管理调度器发送关于操作的信息。 调度器可以应用全局优先级方案来确定执行候选操作的顺序。 该顺序可以基于操作是否包括分区划分或分区移动,无论它们旨在减少存储设备或逻辑卷上的配置存储容量还是减少吞吐量容量,无论它们是否相互冲突,是否是分区的总数( 或其副本)在任何给定时间内涉及分区管理超过预定限制,或者是否被客户请求。
-
公开(公告)号:US11928029B2
公开(公告)日:2024-03-12
申请号:US17019072
申请日:2020-09-11
Applicant: Amazon Technologies, Inc.
Inventor: Maximiliano Maccanti , Timothy Andrew Rath , Rama Krishna Sandeep Pokkunuri , Akshat Vig , Clarence Wing Yin Ng , Srivaths Badrinath Copparam , Rajaprabhu Thiruchi Loganathan , Wei Xiao , William Alexander Stevenson
CPC classification number: G06F11/1451 , G06F11/1464 , G06F11/1469 , G06F11/1458 , G06F11/2094 , G06F2201/80
Abstract: A system that implements a data storage service may store data for a database table in multiple replicated partitions on respective storage nodes. In response to a request to back up a table, the service may back up individual partitions of the table to a remote storage system independently and (in some cases) in parallel, and may update (or create) and store metadata about the table and its partitions on storage nodes of the data storage service and/or in the remote storage system. Backing up each partition may include exporting it from the database in which the table is stored, packaging and compressing the exported partition for upload, and uploading the exported, packaged, and compressed partition to the remote storage system. The remote storage system may be a key-value durable storage system in which each backed-up partition is accessible using its partition identifier as the key.
-
公开(公告)号:US11841844B2
公开(公告)日:2023-12-12
申请号:US13898151
申请日:2013-05-20
Applicant: Amazon Technologies, Inc.
Inventor: Wei Xiao , Clarence Wing Yin Ng , Medhavi Dhawan , Timothy Andrew Rath , Stefano Stefani
IPC: G06F16/23
CPC classification number: G06F16/2358
Abstract: Distributed database management systems may maintain collections of items spanning multiple partitions. Index structures may correspond to items on one partition or to items on multiple partitions. Item collections and indexes may be replicated. Changes to the data maintained by the distributed database management system may result in updates to multiple index structures. The changes may be compiled into an instruction set applicable to the index structures. In-memory buffers may contain the instructions prior to transmission to affected partitions. Replication logs may be combined with an acknowledgment mechanism for reliable transmission of the instructions to the affected partitions.
-
公开(公告)号:US10776212B2
公开(公告)日:2020-09-15
申请号:US15495827
申请日:2017-04-24
Applicant: Amazon Technologies, Inc.
Inventor: Maximliano Maccanti , Timothy Andrew Rath , Rama Krishna Sandeep Pokkunuri , Akshat Vig , Clarence Wing Yin Ng , Srivaths Badrinath Copparam , Rajaprabhu Thiruchi Loganathan , Wei Xiao , William Alexander Stevenson
Abstract: A system that implements a data storage service may store data for a database table in multiple replicated partitions on respective storage nodes. In response to a request to back up a table, the service may back up individual partitions of the table to a remote storage system independently and (in some cases) in parallel, and may update (or create) and store metadata about the table and its partitions on storage nodes of the data storage service and/or in the remote storage system. Backing up each partition may include exporting it from the database in which the table is stored, packaging and compressing the exported partition for upload, and uploading the exported, packaged, and compressed partition to the remote storage system. The remote storage system may be a key-value durable storage system in which each backed-up partition is accessible using its partition identifier as the key.
-
公开(公告)号:US11327949B2
公开(公告)日:2022-05-10
申请号:US15495815
申请日:2017-04-24
Applicant: Amazon Technologies, Inc.
Inventor: Maximiliano Maccanti , Timothy Andrew Rath , Rama Krishna Sandeep Pokkunuri , Akshat Vig , Clarence Wing Yin Ng , Srivaths Badrinath Copparam , Rajaprabhu Thiruchi Loganathan , Wei Xiao , William Alexander Stevenson
Abstract: A system that implements a data storage service may store data for database tables in multiple replicated partitions on respective storage nodes. In response to a request to back up a table, the service may export individual partitions of the table from the database and package them to be independently uploaded (e.g., in parallel) to a remote storage system (e.g., a key-value durable storage system). Prior to uploading the exported and packaged partitions to the remote storage system, the service may verify that the exported and packaged partitions can be subsequently restored, which may include unpackaging and/or re-inflating the exported and packaged partitions to create additional unpackaged copies of the partitions, re-importing the additional unpackaged copies of the partitions into the database (e.g., as additional replicas), and/or comparing checksums generated for the exported partitions with checksums generated for the additional unpackaged copies of the partitions.
-
公开(公告)号:US10025673B1
公开(公告)日:2018-07-17
申请号:US14032870
申请日:2013-09-20
Applicant: Amazon Technologies, Inc.
Inventor: Maximiliano Maccanti , Timothy Andrew Rath , Rama Krishna Sandeep Pokkunuri , Akshat Vig , Clarence Wing Yin Ng , Srivaths Badrinath Copparam , Rajaprabhu Thiruchi Loganathan , Wei Xiao , William Alexander Stevenson
Abstract: A system that implements a data storage service may store data for database tables in multiple replicated partitions on respective storage nodes. In response to a request to restore a given table that was backed up in a remote storage system (e.g., key-value durable storage system), the service may create a new table, and may import a copy of each of the partitions of the given table from the remote storage system into the new table. The request to restore the table may specify a modified value for a configuration parameter for the table or for one of its partitions. The service may apply the new configuration parameter value to the table or its partitions during the restore operation. The new configuration parameter value may indicate an increase or decrease in storage capacity or throughput capacity, and its application may automatically trigger a partition split or move operation.
-
公开(公告)号:US09632878B1
公开(公告)日:2017-04-25
申请号:US14032894
申请日:2013-09-20
Applicant: Amazon Technologies, Inc.
Inventor: Maximiliano Maccanti , Timothy Andrew Rath , Rama Krishna Sandeep Pokkunuri , Akshat Vig , Clarence Wing Yin Ng , Srivaths Badrinath Copparam , Rajaprabhu Thiruchi Loganathan , Wei Xiao , William Alexander Stevenson
CPC classification number: G06F11/1469 , G06F11/1464 , G06F11/2094 , G06F2201/80
Abstract: A system that implements a data storage service may store data for database tables in multiple replicated partitions on respective storage nodes. In response to a request to back up a table, the service may export individual partitions of the table from the database and package them to be independently uploaded (e.g., in parallel) to a remote storage system (e.g., a key-value durable storage system). Prior to uploading the exported and packaged partitions to the remote storage system, the service may verify that the exported and packaged partitions can be subsequently restored, which may include unpackaging and/or re-inflating the exported and packaged partitions to create additional unpackaged copies of the partitions, re-importing the additional unpackaged copies of the partitions into the database (e.g., as additional replicas), and/or comparing checksums generated for the exported partitions with checksums generated for the additional unpackaged copies of the partitions.
-
公开(公告)号:US20140344236A1
公开(公告)日:2014-11-20
申请号:US13898151
申请日:2013-05-20
Applicant: Amazon Technologies, Inc.
Inventor: Wei Xiao , Clarence Wing Yin Ng , Medhavi Dhawan , Timothy Andrew Rath , Stefano Stefani
IPC: G06F17/30
Abstract: Distributed database management systems may maintain collections of items spanning multiple partitions. Index structures may correspond to items on one partition or to items on multiple partitions. Item collections and indexes may be replicated. Changes to the data maintained by the distributed database management system may result in updates to multiple index structures. The changes may be compiled into an instruction set applicable to the index structures. In-memory buffers may contain the instructions prior to transmission to affected partitions. Replication logs may be combined with an acknowledgment mechanism for reliable transmission of the instructions to the affected partitions.
Abstract translation: 分布式数据库管理系统可以维护跨越多个分区的项目的集合。 索引结构可以对应于一个分区上的项目或多个分区上的项目。 项目集合和索引可能被复制。 由分布式数据库管理系统维护的数据的更改可能会导致对多个索引结构的更新。 这些更改可以编译成适用于索引结构的指令集。 内存缓冲区可能会在传输到受影响的分区之前包含指令。 复制日志可以与确认机制组合,以将指令可靠地传输到受影响的分区。
-
-
-
-
-
-
-
-
-