-
公开(公告)号：US10852996B2
公开(公告)日：2020-12-01
申请号：US15673271
申请日：2017-08-09
Applicant: Amazon Technologies, Inc.
Inventor： Jianhua Fan , Benjamin Arthur Hawks , Norbert Paul Kusters , Nachiappan Arumugam , Danny Wei , John Luther Guthrie, II
Abstract: A slave storage is provisioned using metadata of a master B-tree and updates to references (e.g., offsets) pertaining to data operations of the master B-tree. Master-slave pairs can be used to provide data redundancy, and a master copy can include the master B-tree with references to corresponding data. When provisioning a slave copy, the master sends a B-tree copy to the slave, which stores the slave B-tree copy, allocates the necessary space on local storage, and updates respective offsets of the slave B-tree copy to point to the local storage. Data from the master can then be transferred to the slave and stored according to a note and commit process that ensures operational sequence of the data. Operations received to the master during the process can be committed to the slave copy until the slave is consistent with the master and able to take over as master in the event of a failure.
-
公开(公告)号：US09753813B1
公开(公告)日：2017-09-05
申请号：US14866655
申请日：2015-09-25
Applicant: Amazon Technologies, Inc.
Inventor： Jianhua Fan , Benjamin Arthur Hawks , Norbert Paul Kusters , Nachiappan Arumugam , Danny Wei , John Luther Guthrie, II
CPC classification number: G06F11/1448 , G06F3/0605 , G06F3/0619 , G06F3/065 , G06F3/067 , G06F3/0689 , G06F11/1464 , G06F11/1471 , G06F2201/84
Abstract: Persistent storage for a master copy is provided using operation numbers. A master copy can include a persistent key-value store such as a B-tree with references to corresponding data. When provisioning a slave copy, the master copy sends a point-in-time copy of the B-tree to the slave copy, which stores a copy of the B-tree, allocates the necessary space, and updates the references of the B-tree to point to a local storage before the data is transferred. When writing the data to persistent storage, a snapshot created on the master copy is an operation that is replicated to the slave copy. The snapshot is generated using a volume view that includes changes to chunks of data of the master copy since a previous snapshot, as determined using the operation number for the previous snapshot. Data (and metadata) for the snapshot is written to persistent storage while new EO operations are processed.
-
公开(公告)号：US11188469B2
公开(公告)日：2021-11-30
申请号：US16706513
申请日：2019-12-06
Applicant: Amazon Technologies, Inc.
Inventor： Danny Wei , John Luther Guthrie, II , James Michael Thompson , Benjamin Arthur Hawks , Norbert P. Kusters
IPC: G06F12/0866 , G06F11/14 , G06F11/34 , G06F12/0868 , G06F12/0804
Abstract: A block-based storage system may implement page cache write logging. Write requests for a data volume maintained at a storage node may be received at a storage node. A page cache for may be updated in accordance with the request. A log record describing the page cache update may be stored in a page cache write log maintained in a persistent storage device. Once the write request is performed in the page cache and recorded in a log record in the page cache write log, the write request may be acknowledged. Upon recovery from a system failure where data in the page cache is lost, log records in the page cache write log may be replayed to restore to the page cache a state of the page cache prior to the system failure.
-
公开(公告)号：US10802921B2
公开(公告)日：2020-10-13
申请号：US16259571
申请日：2019-01-28
Applicant: Amazon Technologies, Inc.
Inventor： Jianhua Fan , Benjamin Arthur Hawks , Norbert Paul Kusters , Nachiappan Arumugam , Danny Wei , John Luther Guthrie
Abstract: Systems and methods for provisioning a slave copy for redundant data storage and for writing data to persistent storage in a block-based storage system using sequential operation numbers are provided. In one embodiment, the method includes maintaining a master copy and a slave copy of a data volume, the master copy including data generated by a plurality of operations having respective sequential operation numbers, receiving a write instruction for second data to be added to the master copy, and recording the second data as a note that is not readable. The method may further include sending a copy of the note from the master copy to the slave copy, committing the note to the master copy with a sequential operation number, and committing the copy of the note to the slave copy based in part on the sequential operation number. A B-tree may be created based at least in part on an offset for a write instruction associated with the second data, a length, and an operation number included in the note.
-
公开(公告)号：US10055352B2
公开(公告)日：2018-08-21
申请号：US14205067
申请日：2014-03-11
Applicant: Amazon Technologies, Inc.
Inventor： Danny Wei , John Luther Guthrie, II , James Michael Thompson , Benjamin Arthur Hawks , Norbert P. Kusters
IPC: G06F12/0866 , G06F11/14
CPC classification number: G06F12/0866 , G06F11/14 , G06F11/1471 , G06F11/3409 , G06F11/3485 , G06F12/0804 , G06F12/0868 , G06F2201/885 , G06F2212/1016 , G06F2212/1032 , G06F2212/313 , G06F2212/461
Abstract: A block-based storage system may implement page cache write logging. Write requests for a data volume maintained at a storage node may be received at a storage node. A page cache for may be updated in accordance with the request. A log record describing the page cache update may be stored in a page cache write log maintained in a persistent storage device. Once the write request is performed in the page cache and recorded in a log record in the page cache write log, the write request may be acknowledged. Upon recovery from a system failure where data in the page cache is lost, log records in the page cache write log may be replayed to restore to the page cache a state of the page cache prior to the system failure.
-
Patent Agency Ranking
公开(公告)号：US10565501B1
公开(公告)日：2020-02-18
申请号：US13866789
申请日：2013-04-19
Applicant: Amazon Technologies, Inc.
Inventor： Marc Stephen Olson , James Michael Thompson , Benjamin Arthur Hawks
IPC: G06N5/02
Abstract: Techniques are described for formally expressing whether sequences of operations performed on block storage devices are sequential or random. In embodiments, determinations of whether these sequences of operations are sequential or random may be used to predict latencies involved with running particular workloads, and to predict representative workloads for particular latencies.
-
公开(公告)号：US10452680B1
公开(公告)日：2019-10-22
申请号：US14866659
申请日：2015-09-25
Applicant: Amazon Technologies, Inc.
Inventor： Jianhua Fan , Benjamin Arthur Hawks , Norbert Paul Kusters , Nachiappan Arumugam , Danny Wei , John Luther Guthrie, II
IPC: G06F16/27 , G06F16/955 , G06F16/22
Abstract: Master-slave pairs can be used to provide data redundancy in an electronic data environment. A master peer can include a B-tree with references to the corresponding data. When provisioning a slave, the master can send a point-in-time copy of the B-tree to the slave, which can allocate the necessary space on local storage and update the references of the B-tree to point to the local storage for the slave. If the master and slave become disconnected, one of the peers can function as a solo master until the peers are again connected, at which point the old peer can be brought current or a new slave provisioned. A log peer can also be provisioned by a solo master, which can store data for operations received during the disconnect for use in catching up a slave peer, which could be the old slave, the log peer, or a new peer.
-
公开(公告)号：US20180357173A1
公开(公告)日：2018-12-13
申请号：US16105481
申请日：2018-08-20
Applicant: Amazon Technologies, Inc.
Inventor： Danny Wei , John Luther Guthrie, II , James Michael Thompson , Benjamin Arthur Hawks , Norbert P. Kusters
IPC: G06F12/0866 , G06F11/14 , G06F12/0868
CPC classification number: G06F12/0866 , G06F11/14 , G06F11/1471 , G06F11/3409 , G06F11/3485 , G06F12/0804 , G06F12/0868 , G06F2201/885 , G06F2212/1016 , G06F2212/1032 , G06F2212/313 , G06F2212/461
Abstract: A block-based storage system may implement page cache write logging. Write requests for a data volume maintained at a storage node may be received at a storage node. A page cache for may be updated in accordance with the request. A log record describing the page cache update may be stored in a page cache write log maintained in a persistent storage device. Once the write request is performed in the page cache and recorded in a log record in the page cache write log, the write request may be acknowledged. Upon recovery from a system failure where data in the page cache is lost, log records in the page cache write log may be replayed to restore to the page cache a state of the page cache prior to the system failure.
-
公开(公告)号：US20170364411A1
公开(公告)日：2017-12-21
申请号：US15694684
申请日：2017-09-01
Applicant: Amazon Technologies, Inc.
Inventor： Jianhua Fan , Benjamin Arthur Hawks , Norbert Paul Kusters , Nachiappan Arumugam , Danny Wei , John Luther Guthrie, II
CPC classification number: G06F11/1448 , G06F3/0605 , G06F3/0619 , G06F3/065 , G06F3/067 , G06F3/0689 , G06F11/1464 , G06F11/1471 , G06F2201/84
Abstract: The present disclosure provides persistent storage for a master copy using operation numbers. A master copy can include a B-tree with references to corresponding data. When provisioning a slave copy, the master copy sends a point-in-time copy of the B-tree to the slave copy, which stores a copy of the B-tree, allocates the necessary space, and updates the references of the B-tree to point to a local storage before the data is transferred. When writing the data to persistent storage, a snapshot created on the master copy is an operation that is replicated to the slave copy. The snapshot is generated using a volume view that includes changes to chunks of data of the master copy since a previous snapshot, as determined using the operation number for the previous snapshot. Data (and metadata) for the snapshot is written to persistent storage while new I/O operations are processed.
-
公开(公告)号：US09792231B1
公开(公告)日：2017-10-17
申请号：US14571183
申请日：2014-12-15
Applicant: Amazon Technologies, Inc.
Inventor： James Michael Thompson , Marc Stephen Olson , Jeevan Shankar , Danny Wei , John Robert Smiley , John Luther Guthrie, II , Nachiappan Arumugam , Benjamin Arthur Hawks
CPC classification number: G06F13/1642 , G06F9/5061 , H04L43/028 , H04L67/10
Abstract: Systems and methods are described for dynamically detecting outliers in a set of input/output (I/O) metrics collected and aggregated by a storage volume network. An I/O request is received by a storage volume network, and an agent of the storage volume network associates primary and secondary identifiers with that I/O request. For example, a trace may be associated with a request to write data to a storage volume network, and spans may be associated with the individual operations required to fulfill that request. Once gathered, I/O metrics may be aggregated based on the associated identifiers. I/O metric information regarding outliers may be received from the storage volume network, processed, and published by an I/O metrics service to identify the outliers among the primary and secondary identifiers. These outliers may then be stored for further analysis, and may be utilized to determine improvements to the performance of a storage volume network.
-
-
-
-
-
-
-
-
-
Search Results
22
records
(took 0.021 seconds)
