摘要:
Methods, apparatus, and computer-accessible storage media for optimizing block storage I/O operations in a storage gateway. A write log may be implemented in a block store as a one-dimensional queue. A read cache may also be implemented in the block store. When non-ordered writes are received, sequential writes may be performed to the write log and the data may be written to contiguous locations on the storage. A metadata store may store metadata for the write log and the read cache. Reads may be satisfied from the write log if possible, or from the read cache or backend store if not. If blocks are read from the read cache or backend store to satisfy a read, the blocks may be mutated with data from the write log before being sent to the requesting process. The mutated blocks may be stored to the read cache.
摘要:
Methods, apparatus, and computer-accessible storage media for restoring data from a snapshot to a data volume. The blocks in the volume may be treated as an implicit tree structure, for example a binary tree; each local block corresponds to a block on the snapshot. A local block on the volume may be marked, for example fingerprinted with metadata, to indicate that the local block has not been restored. Initially, the local block at the root node is marked. To restore a local block, the restore process may generate a list indicating all local blocks on a path from the root node of the tree to the target node that have not been restored. The marks in the local blocks are used in generating the list. For each block indicated in the list, children of the block are fingerprinted, and the block is restored from the snapshot.
摘要:
Methods, apparatus, and computer-accessible storage media for remotely updating an executing process that receives I/O requests on I/O port(s), stores write data to a write log on local storage, and uploads data from the write log to remote storage. An update for the process is detected and downloaded, and an updated process is instantiated from the update. The current process is directed to perform a shutdown for update during an update window. In response, the current process saves its current configuration, flushes an in-memory portion of the write log to local storage, and releases its I/O port(s). The updated process loads the saved configuration, detects that the port(s) have been released, and starts accepting I/O requests on the ports. During flushing, the current process flushes current data in memory while continuing to append new write data, stops accepting new write requests, and then flushes the new write data.
摘要:
Methods, apparatus, and computer-accessible storage media for shadowing data stored on a local store to a remote store provided by a service provider. A gateway may be configured as a shadowing gateway on a customer network in response to receiving configuration information. The shadowing gateway may receive reads and writes to the local store. The gateway passes the requests to the local store, and also uploads write data indicated by the writes to the service provider to update a snapshot of the local store maintained by the service provider on the remote store. The write data may be buffered to a write log for uploading, and may be uploaded as blocks according to a block storage format used by the service provider. The shadowing process may be transparent to processes on the customer network. The shadowed data may be used to recover data on the local store.
摘要:
Methods, apparatus, and computer-accessible storage media for providing redundant storage gateways. A client may create a storage gateway group and add storage gateways to the group. The client may assign one or more volumes on a remote data store to each the storage gateways in the group. Volume data for each storage gateway in the group may be replicated to at least one other storage gateway in the group. If one of the gateways in the group becomes unavailable, one or more other gateways in the group may take over volumes previously assigned to the unavailable gateway, using the replicated data in the group to seamlessly resume gateway operations for the respective volumes. Client processes that previously communicated with the unavailable gateway may be manually or automatically directed to the gateway(s) that are taking over the unavailable gateway's volumes.
摘要:
Methods, apparatus, and computer-accessible storage media for remotely managing a gateway that serves as an interface between processes on a customer network and a service provider, for example to store data to a remote data store. The gateway sends a connection request to a gateway control server. The server holds the connection until the server receives information (e.g., information from the customer sent via the service provider) for the gateway. The server sends the information as requests via the gateway-initiated connection, and continues to hold the connection. If a server receives information for a gateway to which it does not hold a connection, the server sends the information to the server that does hold the connection. The server may either discover the appropriate server via a registration service that registers connections to gateways or broadcast the information to peer servers identified through a registration service.
摘要:
Methods, apparatus, and computer-accessible storage media for remotely updating an executing process that receives I/O requests on I/O port(s), stores write data to a write log on local storage, and uploads data from the write log to remote storage. An update for the process is detected and downloaded, and an updated process is instantiated from the update. The current process is directed to perform a shutdown for update during an update window. In response, the current process saves its current configuration, flushes an in-memory portion of the write log to local storage, and releases its I/O port(s). The updated process loads the saved configuration, detects that the port(s) have been released, and starts accepting I/O requests on the ports. During flushing, the current process flushes current data in memory while continuing to append new write data, stops accepting new write requests, and then flushes the new write data.
摘要:
Methods, apparatus, and computer-accessible storage media for shadowing data stored on a local store to a remote store provided by a service provider. A gateway may be configured as a shadowing gateway on a customer network in response to receiving configuration information. The shadowing gateway may receive reads and writes to the local store. The gateway passes the requests to the local store, and also uploads write data indicated by the writes to the service provider to update a snapshot of the local store maintained by the service provider on the remote store. The write data may be buffered to a write log for uploading, and may be uploaded as blocks according to a block storage format used by the service provider. The shadowing process may be transparent to processes on the customer network. The shadowed data may be used to recover data on the local store.
摘要:
Methods, apparatus, and computer-accessible storage media for remotely monitoring and diagnosing storage gateways. Status information may be collected locally on the gateways and uploaded to a service provider via gateway-initiated connections. The uploaded information may be stored to status data store(s). Status proxy(s) on the provider network may analyze the information in the status data store(s) for one or more gateways to detect error conditions on individual gateways or patterns or error conditions on multiple gateways. Upon detecting an error condition on a gateway, the proxy may alert another process, for example an administrator process on the local network that includes the respective gateway. The other process may then message the gateway to address the condition. Information for particular gateways may be provided to clients on request. Information collected from multiple gateways may be viewed and analyzed by the service provider to detect patterns related to gateway design.
摘要:
Methods, apparatus, and computer-accessible storage media for implementing a gateway to a remote service provider according to a security model. The gateway serves as an interface between processes on a customer network and the provider, for example to store customer data to a remote data store. The model may include an activation process initiated by the gateway to register with the provider and associate the gateway with a customer account; the gateway is provided with security credentials. The model may also include establishing secure connections to external processes, for example processes of the service provider. The gateway initiates connections; the external processes do not initiate connections. The model may also include the customer managing the gateway through the service provider. The model may also include encrypting communications between the gateway and the provider and the gateway including security credentials in communications to the provider.