摘要:
Methods, apparatus, and computer-accessible storage media for optimizing block storage I/O operations in a storage gateway. A write log may be implemented in a block store as a one-dimensional queue. A read cache may also be implemented in the block store. When non-ordered writes are received, sequential writes may be performed to the write log and the data may be written to contiguous locations on the storage. A metadata store may store metadata for the write log and the read cache. Reads may be satisfied from the write log if possible, or from the read cache or backend store if not. If blocks are read from the read cache or backend store to satisfy a read, the blocks may be mutated with data from the write log before being sent to the requesting process. The mutated blocks may be stored to the read cache.
摘要:
Methods, apparatus, and computer-accessible storage media for activating a gateway to a remote service provider. The gateway serves as an interface between processes on a customer network and the provider, for example to store customer data to a remote data store. A gateway sends a public key and metadata describing the gateway to the provider. The gateway receives an activation key from the provider and exposes the activation key on the customer network. The customer obtains the key and communicates to the provider using the key to provide customer information including a name for the gateway and to authorize registration of the gateway. The provider provides the customer information to the gateway. The gateway requests security credentials from the provider using the customer information and the key. The provider sends a security credential to the gateway. The gateway may then obtain configuration information from the customer via the provider.
摘要:
Methods, apparatus, and computer-accessible storage media for activating a gateway to a remote service provider. The gateway serves as an interface between processes on a customer network and the provider, for example to store customer data to a remote data store. A gateway sends a public key and metadata describing the gateway to the provider. The gateway receives an activation key from the provider and exposes the activation key on the customer network. The customer obtains the key and communicates to the provider using the key to provide customer information including a name for the gateway and to authorize registration of the gateway. The provider provides the customer information to the gateway. The gateway requests security credentials from the provider using the customer information and the key. The provider sends a security credential to the gateway. The gateway may then obtain configuration information from the customer via the provider.
摘要:
Methods, apparatus, and computer-accessible storage media for remotely managing a gateway that serves as an interface between processes on a customer network and a service provider, for example to store data to a remote data store. The gateway sends a connection request to a gateway control server. The server holds the connection until the server receives information (e.g., information from the customer sent via the service provider) for the gateway. The server sends the information as requests via the gateway-initiated connection, and continues to hold the connection. If a server receives information for a gateway to which it does not hold a connection, the server sends the information to the server that does hold the connection. The server may either discover the appropriate server via a registration service that registers connections to gateways or broadcast the information to peer servers identified through a registration service.
摘要:
Methods, apparatus, and computer-accessible storage media for shadowing data stored on a local store to a remote store provided by a service provider. A gateway may be configured as a shadowing gateway on a customer network in response to receiving configuration information. The shadowing gateway may receive reads and writes to the local store. The gateway passes the requests to the local store, and also uploads write data indicated by the writes to the service provider to update a snapshot of the local store maintained by the service provider on the remote store. The write data may be buffered to a write log for uploading, and may be uploaded as blocks according to a block storage format used by the service provider. The shadowing process may be transparent to processes on the customer network. The shadowed data may be used to recover data on the local store.
摘要:
Methods, apparatus, and computer-accessible storage media for implementing a gateway to a remote service provider according to a security model. The gateway serves as an interface between processes on a customer network and the provider, for example to store customer data to a remote data store. The model may include an activation process initiated by the gateway to register with the provider and associate the gateway with a customer account; the gateway is provided with security credentials. The model may also include establishing secure connections to external processes, for example processes of the service provider. The gateway initiates connections; the external processes do not initiate connections. The model may also include the customer managing the gateway through the service provider. The model may also include encrypting communications between the gateway and the provider and the gateway including security credentials in communications to the provider.
摘要:
Methods, apparatus, and computer-accessible storage media for shadowing data stored on a local store to a remote store provided by a service provider. A gateway may be configured as a shadowing gateway on a customer network in response to receiving configuration information. The shadowing gateway may receive reads and writes to the local store. The gateway passes the requests to the local store, and also uploads write data indicated by the writes to the service provider to update a snapshot of the local store maintained by the service provider on the remote store. The write data may be buffered to a write log for uploading, and may be uploaded as blocks according to a block storage format used by the service provider. The shadowing process may be transparent to processes on the customer network. The shadowed data may be used to recover data on the local store.
摘要:
Methods, apparatus, and computer-accessible storage media for restoring data from a snapshot to a data volume. The blocks in the volume may be treated as an implicit tree structure, for example a binary tree; each local block corresponds to a block on the snapshot. A local block on the volume may be marked, for example fingerprinted with metadata, to indicate that the local block has not been restored. Initially, the local block at the root node is marked. To restore a local block, the restore process may generate a list indicating all local blocks on a path from the root node of the tree to the target node that have not been restored. The marks in the local blocks are used in generating the list. For each block indicated in the list, children of the block are fingerprinted, and the block is restored from the snapshot.
摘要:
Methods, apparatus, and computer-accessible storage media for remotely updating an executing process that receives I/O requests on I/O port(s), stores write data to a write log on local storage, and uploads data from the write log to remote storage. An update for the process is detected and downloaded, and an updated process is instantiated from the update. The current process is directed to perform a shutdown for update during an update window. In response, the current process saves its current configuration, flushes an in-memory portion of the write log to local storage, and releases its I/O port(s). The updated process loads the saved configuration, detects that the port(s) have been released, and starts accepting I/O requests on the ports. During flushing, the current process flushes current data in memory while continuing to append new write data, stops accepting new write requests, and then flushes the new write data.
摘要:
Methods, apparatus, and computer-accessible storage media for providing redundant storage gateways. A client may create a storage gateway group and add storage gateways to the group. The client may assign one or more volumes on a remote data store to each the storage gateways in the group. Volume data for each storage gateway in the group may be replicated to at least one other storage gateway in the group. If one of the gateways in the group becomes unavailable, one or more other gateways in the group may take over volumes previously assigned to the unavailable gateway, using the replicated data in the group to seamlessly resume gateway operations for the respective volumes. Client processes that previously communicated with the unavailable gateway may be manually or automatically directed to the gateway(s) that are taking over the unavailable gateway's volumes.