-
公开(公告)号:US10469405B2
公开(公告)日:2019-11-05
申请号:US15197557
申请日:2016-06-29
Applicant: Amazon Technologies, Inc.
Inventor: Pieter Kristian Brouwer , Marc Stephen Olson , Nachiappan Arumugam , Michael Thacker , Vijay Prasanth Rajavenkateswaran , Arpit Tripathi , Danny Wei
IPC: H04L12/927
Abstract: A user can set or modify operational parameters of a data volume stored on a network-accessible storage device in a data center. For example, the user may be provided access to a data volume and may request a modification to the operational parameters of the data volume. Instead of modifying the existing data volume, the data center can provision a new data volume and migrate data stored on the existing data volume to the new data volume. While the data migration takes place, the existing data volume may block input/output (I/O) requests and the new data volume may handle such requests instead. Once the data migration is complete, the data center may deallocate the data blocks of the existing data volume such that the data blocks can be reused by other data volumes.
-
公开(公告)号:US10452680B1
公开(公告)日:2019-10-22
申请号:US14866659
申请日:2015-09-25
Applicant: Amazon Technologies, Inc.
Inventor: Jianhua Fan , Benjamin Arthur Hawks , Norbert Paul Kusters , Nachiappan Arumugam , Danny Wei , John Luther Guthrie, II
IPC: G06F16/27 , G06F16/955 , G06F16/22
Abstract: Master-slave pairs can be used to provide data redundancy in an electronic data environment. A master peer can include a B-tree with references to the corresponding data. When provisioning a slave, the master can send a point-in-time copy of the B-tree to the slave, which can allocate the necessary space on local storage and update the references of the B-tree to point to the local storage for the slave. If the master and slave become disconnected, one of the peers can function as a solo master until the peers are again connected, at which point the old peer can be brought current or a new slave provisioned. A log peer can also be provisioned by a solo master, which can store data for operations received during the disconnect for use in catching up a slave peer, which could be the old slave, the log peer, or a new peer.
-
公开(公告)号:US20170364411A1
公开(公告)日:2017-12-21
申请号:US15694684
申请日:2017-09-01
Applicant: Amazon Technologies, Inc.
Inventor: Jianhua Fan , Benjamin Arthur Hawks , Norbert Paul Kusters , Nachiappan Arumugam , Danny Wei , John Luther Guthrie, II
CPC classification number: G06F11/1448 , G06F3/0605 , G06F3/0619 , G06F3/065 , G06F3/067 , G06F3/0689 , G06F11/1464 , G06F11/1471 , G06F2201/84
Abstract: The present disclosure provides persistent storage for a master copy using operation numbers. A master copy can include a B-tree with references to corresponding data. When provisioning a slave copy, the master copy sends a point-in-time copy of the B-tree to the slave copy, which stores a copy of the B-tree, allocates the necessary space, and updates the references of the B-tree to point to a local storage before the data is transferred. When writing the data to persistent storage, a snapshot created on the master copy is an operation that is replicated to the slave copy. The snapshot is generated using a volume view that includes changes to chunks of data of the master copy since a previous snapshot, as determined using the operation number for the previous snapshot. Data (and metadata) for the snapshot is written to persistent storage while new I/O operations are processed.
-
公开(公告)号:US09792231B1
公开(公告)日:2017-10-17
申请号:US14571183
申请日:2014-12-15
Applicant: Amazon Technologies, Inc.
Inventor: James Michael Thompson , Marc Stephen Olson , Jeevan Shankar , Danny Wei , John Robert Smiley , John Luther Guthrie, II , Nachiappan Arumugam , Benjamin Arthur Hawks
CPC classification number: G06F13/1642 , G06F9/5061 , H04L43/028 , H04L67/10
Abstract: Systems and methods are described for dynamically detecting outliers in a set of input/output (I/O) metrics collected and aggregated by a storage volume network. An I/O request is received by a storage volume network, and an agent of the storage volume network associates primary and secondary identifiers with that I/O request. For example, a trace may be associated with a request to write data to a storage volume network, and spans may be associated with the individual operations required to fulfill that request. Once gathered, I/O metrics may be aggregated based on the associated identifiers. I/O metric information regarding outliers may be received from the storage volume network, processed, and published by an I/O metrics service to identify the outliers among the primary and secondary identifiers. These outliers may then be stored for further analysis, and may be utilized to determine improvements to the performance of a storage volume network.
-
公开(公告)号:US11438411B2
公开(公告)日:2022-09-06
申请号:US17013430
申请日:2020-09-04
Applicant: Amazon Technologies, Inc.
Inventor: Norbert P. Kusters , Nachiappan Arumugam , Christopher Nathan Watson , Marc John Brooker , David R. Richardson , Danny Wei , John Luther Guthrie, II
IPC: H04L67/1095 , H04L67/1097 , H04L61/2503 , G06F11/20 , G06F3/06 , H04L67/568 , H04L67/2885 , G06F15/16 , G06F11/10 , H03M13/15 , H04L67/01 , G06F11/16 , G06F11/34
Abstract: A data storage system includes a rack, multiple head nodes, multiple data storage sleds, and at least two networking devices. The at least two network devices are configured to implement at least two redundant networks within the data storage system. Also, each of the head nodes is assigned at least two network addresses for communication with the data storage sleds of the data storage system via the at least two networking devices. The data storage sleds each include multiple mass storage devices and a sled controller that is configured to couple with the at least two network switches. IO In some embodiments, the data storage system further includes redundant power systems within a rack in which the head nodes, the data storage sleds, and the at least two networking devices are mounted.
-
公开(公告)号:US20210263658A1
公开(公告)日:2021-08-26
申请号:US17319020
申请日:2021-05-12
Applicant: Amazon Technologies, Inc.
Inventor: Norbert Paul Kusters , Nachiappan Arumugam , Andre Podnozov , Shobha Agrawal , Shreyas Ramalingam , Danny Wei , David R. Richardson , Marc John Brooker , Christopher Nathan Watson , John Luther Guthrie, II , Ravi Nankani
Abstract: A data storage system includes a head node and mass storage devices. The head node is configured to flush data stored in a storage of the head node to a set of the mass storage devices of the data storage system. A head node may flush both current version data and point-in-time version data to the set of mass storage devices. Also, the data storage system maintains an index that indicates storage locations of data for particular portions of a volume before and after the data is flushed to the set of mass storage devices. In some embodiments an index includes a current version reference for a volume or portion of a volume and one or more point-in-time snapshot references.
-
公开(公告)号:US11010064B2
公开(公告)日:2021-05-18
申请号:US15433956
申请日:2017-02-15
Applicant: Amazon Technologies, Inc.
Inventor: Norbert Paul Kusters , Nachiappan Arumugam , Andre Podnozov , Shobha Agrawal , Shreyas Ramalingam , Danny Wei , David R. Richardson , Marc John Brooker , Christopher Nathan Watson , John Luther Guthrie, II , Ravi Nankani
Abstract: A data storage system includes a head node and mass storage devices. The head node is configured to flush data stored in a storage of the head node to a set of the mass storage devices of the data storage system. A head node may flush both current version data and point-in-time version data to the set of mass storage devices. Also, the data storage system maintains an index that indicates storage locations of data for particular portions of a volume before and after the data is flushed to the set of mass storage devices. In some embodiments an index includes a current version reference for a volume or portion of a volume and one or more point-in-time snapshot references.
-
公开(公告)号:US10852996B2
公开(公告)日:2020-12-01
申请号:US15673271
申请日:2017-08-09
Applicant: Amazon Technologies, Inc.
Inventor: Jianhua Fan , Benjamin Arthur Hawks , Norbert Paul Kusters , Nachiappan Arumugam , Danny Wei , John Luther Guthrie, II
Abstract: A slave storage is provisioned using metadata of a master B-tree and updates to references (e.g., offsets) pertaining to data operations of the master B-tree. Master-slave pairs can be used to provide data redundancy, and a master copy can include the master B-tree with references to corresponding data. When provisioning a slave copy, the master sends a B-tree copy to the slave, which stores the slave B-tree copy, allocates the necessary space on local storage, and updates respective offsets of the slave B-tree copy to point to the local storage. Data from the master can then be transferred to the slave and stored according to a note and commit process that ensures operational sequence of the data. Operations received to the master during the process can be committed to the slave copy until the slave is consistent with the master and able to take over as master in the event of a failure.
-
公开(公告)号:US10812408B1
公开(公告)日:2020-10-20
申请号:US15081646
申请日:2016-03-25
Applicant: Amazon Technologies, Inc.
Inventor: Surya Prakash Dhoolam , Christopher Magee Greenwood , Mitchell Gannon Flaherty , Danny Wei , Nachiappan Arumugam
IPC: H04L12/911
Abstract: A distributed system may implement preventing concentrated selection of resource hosts for placing resources within the distributed system. When a request to place a resource at a resource host is received, an evaluation of load metrics for the resource host is made with respect to rejection criteria. If the rejection criteria is satisfied, then the resource host may reject the request to place the resource. If the rejection criteria is not satisfied, then the resource host may not reject the placement of the resource. Rejection criteria may be provided to resource hosts along with the placement request, in some embodiments. Rejection criteria may be changed, weakened, or removed entirely if initial attempts to place a resource are unsuccessful.
-
公开(公告)号:US10484015B2
公开(公告)日:2019-11-19
申请号:US15392806
申请日:2016-12-28
Applicant: Amazon Technologies, Inc.
Inventor: Norbert P. Kusters , Nachiappan Arumugam , Christopher Nathan Watson , Marc John Brooker , David R. Richardson , Danny Wei , John Luther Guthrie, II , Leah Shalev
Abstract: A data storage system includes multiple head nodes and data storage sleds. The data storage sleds include multiple mass storage devices and a sled controller. Respective ones of the head nodes are configured to obtain credentials for accessing particular portions of the mass storage devices of the data storage sleds. A sled controller of a data storage sled determines whether a head node attempting to perform a write on a mass storage device of a data storage sled that includes the sled controller is presenting with the write request a valid credential for accessing the mass storage devices of the data storage sled. If the credentials are valid, the sled controller causes the write to be performed and if the credentials are invalid, the sled controller returns a message to the head node indicating that it has been fenced off from the mass storage device.
-
-
-
-
-
-
-
-
-