-
公开(公告)号:US20210263658A1
公开(公告)日:2021-08-26
申请号:US17319020
申请日:2021-05-12
Applicant: Amazon Technologies, Inc.
Inventor: Norbert Paul Kusters , Nachiappan Arumugam , Andre Podnozov , Shobha Agrawal , Shreyas Ramalingam , Danny Wei , David R. Richardson , Marc John Brooker , Christopher Nathan Watson , John Luther Guthrie, II , Ravi Nankani
Abstract: A data storage system includes a head node and mass storage devices. The head node is configured to flush data stored in a storage of the head node to a set of the mass storage devices of the data storage system. A head node may flush both current version data and point-in-time version data to the set of mass storage devices. Also, the data storage system maintains an index that indicates storage locations of data for particular portions of a volume before and after the data is flushed to the set of mass storage devices. In some embodiments an index includes a current version reference for a volume or portion of a volume and one or more point-in-time snapshot references.
-
公开(公告)号:US11010064B2
公开(公告)日:2021-05-18
申请号:US15433956
申请日:2017-02-15
Applicant: Amazon Technologies, Inc.
Inventor: Norbert Paul Kusters , Nachiappan Arumugam , Andre Podnozov , Shobha Agrawal , Shreyas Ramalingam , Danny Wei , David R. Richardson , Marc John Brooker , Christopher Nathan Watson , John Luther Guthrie, II , Ravi Nankani
Abstract: A data storage system includes a head node and mass storage devices. The head node is configured to flush data stored in a storage of the head node to a set of the mass storage devices of the data storage system. A head node may flush both current version data and point-in-time version data to the set of mass storage devices. Also, the data storage system maintains an index that indicates storage locations of data for particular portions of a volume before and after the data is flushed to the set of mass storage devices. In some embodiments an index includes a current version reference for a volume or portion of a volume and one or more point-in-time snapshot references.
-
公开(公告)号:US10929041B1
公开(公告)日:2021-02-23
申请号:US16560860
申请日:2019-09-04
Applicant: Amazon Technologies, Inc.
Inventor: Fan Ping , Andrew Boyer , Oleksandr Chychykalo , James Pinkerton , Danny Wei , Norbert Paul Kusters , Divya Ashok Kumar Jain , Jianhua Fan , Thomas Tarak Mathew Veppumthara , Sebastiano Peluso
Abstract: A block-based storage system hosts logical volumes that are implemented via multiple replicas of volume data stored on multiple resource hosts in different failure domains. Also, the block-based storage service allows multiple client computing devices to attach to a same given logical volume at the same time. A membership group authority authorizes sequence numbers for a given logical volume and an associated membership group. The members of the membership group ensure that the members are in agreement on the latest sequence number for the given logical volume before responding to read or write requests directed to the given logical volume.
-
公开(公告)号:US10852996B2
公开(公告)日:2020-12-01
申请号:US15673271
申请日:2017-08-09
Applicant: Amazon Technologies, Inc.
Inventor: Jianhua Fan , Benjamin Arthur Hawks , Norbert Paul Kusters , Nachiappan Arumugam , Danny Wei , John Luther Guthrie, II
Abstract: A slave storage is provisioned using metadata of a master B-tree and updates to references (e.g., offsets) pertaining to data operations of the master B-tree. Master-slave pairs can be used to provide data redundancy, and a master copy can include the master B-tree with references to corresponding data. When provisioning a slave copy, the master sends a B-tree copy to the slave, which stores the slave B-tree copy, allocates the necessary space on local storage, and updates respective offsets of the slave B-tree copy to point to the local storage. Data from the master can then be transferred to the slave and stored according to a note and commit process that ensures operational sequence of the data. Operations received to the master during the process can be committed to the slave copy until the slave is consistent with the master and able to take over as master in the event of a failure.
-
公开(公告)号:US10705956B1
公开(公告)日:2020-07-07
申请号:US15969604
申请日:2018-05-02
Applicant: Amazon Technologies, Inc.
Inventor: Kristina Kraemer Brenneman , Norbert Paul Kusters , Jianhua Fan , Danny Wei
IPC: G06F12/08 , G06F12/0804 , G06F9/52 , G06F16/23
Abstract: A data storage system stores information indicating a determined sequence for performing operations on a data store. A lock is acquired on a portion of the data store. It is determined that performing the operations comprises performing at least one additional operation on the data store. Uncommitted changes implied by the operations are stored in a transaction buffer according to the determined sequence. Changes implied by the additional operation are determined based on a reentrant call to a data store interface. The logged sequence of changes is applied to the data store and the lock is released.
-
公开(公告)号:US20190324812A1
公开(公告)日:2019-10-24
申请号:US16460914
申请日:2019-07-02
Applicant: Amazon Technologies, Inc.
Inventor: Norbert Paul Kusters , John Robert Smiley , Marc John Brooker , Bei-Jing Guo , Marc Levy
Abstract: Burst throttling methods may be used to manage computing resources of a data storage service. Tokens may represent I/O operations executed by a customer of the data storage service. A first token bucket may contain a set of tokens representing the overall I/O operation capacity of the data storage service. Additionally, a second token bucket may contain a set of tokens for a given logical volume maintained by the data storage service. When I/O requests are received tokens may be charged the first token bucket and the second token bucket. Furthermore, if there is sufficient capacity, the data storage service may charge a reduced number of tokens to the third token bucket.
-
公开(公告)号:US20190324666A1
公开(公告)日:2019-10-24
申请号:US16457095
申请日:2019-06-28
Applicant: Amazon Technologies, Inc.
Inventor: Norbert Paul Kusters , Jianhua Fan , Shuvabrata Ganguly , Danny Wei , Avram Israel Blaszka
IPC: G06F3/06
Abstract: A data storage system includes multiple head nodes and data storage sleds. A control plane of the data storage system designates, for a volume partition, one of the head nodes to function as a primary head node storing a primary replica of the volume partition and designates two or more other head nodes to function as reserve head nodes storing reserve replicas of the volume partition. Additionally, the primary head node causes volume data for the volume partition to be erasure encoded and stored on multiple mass storage devices in different ones of the data storage sleds.
-
公开(公告)号:US09753813B1
公开(公告)日:2017-09-05
申请号:US14866655
申请日:2015-09-25
Applicant: Amazon Technologies, Inc.
Inventor: Jianhua Fan , Benjamin Arthur Hawks , Norbert Paul Kusters , Nachiappan Arumugam , Danny Wei , John Luther Guthrie, II
CPC classification number: G06F11/1448 , G06F3/0605 , G06F3/0619 , G06F3/065 , G06F3/067 , G06F3/0689 , G06F11/1464 , G06F11/1471 , G06F2201/84
Abstract: Persistent storage for a master copy is provided using operation numbers. A master copy can include a persistent key-value store such as a B-tree with references to corresponding data. When provisioning a slave copy, the master copy sends a point-in-time copy of the B-tree to the slave copy, which stores a copy of the B-tree, allocates the necessary space, and updates the references of the B-tree to point to a local storage before the data is transferred. When writing the data to persistent storage, a snapshot created on the master copy is an operation that is replicated to the slave copy. The snapshot is generated using a volume view that includes changes to chunks of data of the master copy since a previous snapshot, as determined using the operation number for the previous snapshot. Data (and metadata) for the snapshot is written to persistent storage while new EO operations are processed.
-
公开(公告)号:US20230359383A1
公开(公告)日:2023-11-09
申请号:US18297403
申请日:2023-04-07
Applicant: Amazon Technologies, Inc.
Inventor: Swapnil Vinay Dinkar , Pradeep Kunni Raman , David Matthew Buches , Hon Ping Shea , Norbert Paul Kusters
IPC: G06F3/06
CPC classification number: G06F3/0644 , G06F3/0631 , G06F3/0689 , G06F3/0665 , G06F3/0604 , G06F3/064
Abstract: A storage service supports attachment of multiple clients to a distributed storage object and further supports persistent reservations that govern types of access the respective clients are granted with respect to the distributed storage object. In order to efficiently distribute reservation state changes to multiple partitions of the distributed storage object hosted by different data storage units/servers, existing connections are used between the data storage units/servers hosting the partitions of the distributed storage object and the connected clients to propagate reservation state changes.
-
公开(公告)号:US20220057951A1
公开(公告)日:2022-02-24
申请号:US17520537
申请日:2021-11-05
Applicant: Amazon Technologies, Inc.
Inventor: Norbert Paul Kusters , Jianhua Fan , Shuvabrata Ganguly , Danny Wei , Avram Israel Blaszka
Abstract: A data storage system includes multiple head nodes and data storage sleds. Volume data is replicated between a primary and one or more secondary head nodes for a volume partition and is further flushed to a set of mass storage devices of the data storage sleds. Volume metadata is maintained in a primary and one or more secondary head nodes for a volume partition and is updated in response to volume data being flushed to the data storage sleds. Also, the primary and secondary head nodes store check-points of volume metadata to the data storage sleds, wherein in response to a failure of a primary or secondary head node for a volume partition, a replacement secondary head node for the volume partition recreates a secondary replica for the volume partition based, at least in part, on a stored volume metadata checkpoint.
-
-
-
-
-
-
-
-
-