公开(公告)号：US20120303577A1

公开(公告)日：2012-11-29

申请号：US13113688

申请日：2011-05-23

申请人： Bradley Gene Calder ,  Niranjan Nilakantan ,  Shashwat Srivastav ,  Jiesheng Wu ,  Abdul Rafay Abbasi ,  Shane Mainali ,  Padmanabha Chakravarthy Uddaraju

发明人： Bradley Gene Calder ,  Niranjan Nilakantan ,  Shashwat Srivastav ,  Jiesheng Wu ,  Abdul Rafay Abbasi ,  Shane Mainali ,  Padmanabha Chakravarthy Uddaraju

IPC分类号： G06F7/00

CPC分类号： G06F17/30578 ,  G06F11/2074 ,  G06F11/2097 ,  G06F17/30241 ,  G06F17/30584 ,  G06F17/30589 ,  G06F2201/855

摘要： Embodiments of the present invention relate to asynchronously replicating data in a distributed computing environment. To achieve asynchronous replication, data received at a primary data store may be annotated with information, such as an identifier of the data. The annotated data may then be communicated to a secondary data store, which may then write the data and annotated information to one or more logs for eventual replay and committal at the secondary data store. The primary data store may communicate an acknowledgment of success in committing the data at the primary data store as well as of success in writing the data to the secondary data store. Additional embodiments may include committing the data at the secondary data store in response to receiving an instruction that authorizes committal of data through a identifier.

摘要翻译： 本发明的实施例涉及在分布式计算环境中异步复制数据。 为了实现异步复制，在主数据存储处接收到的数据可以用诸如数据的标识符的信息来注释。 然后可以将注释的数据传送到辅助数据存储器，然后辅助数据存储器可以将数据和注释的信息写入一个或多个日志以在次要数据存储器上最终重放和提交。 主数据存储器可以在主数据存储器上传送提交数据的成功确认以及将数据写入辅助数据存储器的成功。 另外的实施例可以包括在辅助数据存储器处响应于接收授权通过标识符提交数据的指令来提交数据。

公开(公告)号：US20120303912A1

公开(公告)日：2012-11-29

申请号：US13113808

申请日：2011-05-23

申请人： Bradley Gene Calder ,  Niranjan Nilakantan ,  Shashwat Srivastav ,  Jiesheng Wu ,  Arild Einar Skjolsvold ,  Maxim Mazeev ,  Abdul Rafay Abbasi ,  Shane Mainali ,  Hemal Khatri ,  Ju Wang

发明人： Bradley Gene Calder ,  Niranjan Nilakantan ,  Shashwat Srivastav ,  Jiesheng Wu ,  Arild Einar Skjolsvold ,  Maxim Mazeev ,  Abdul Rafay Abbasi ,  Shane Mainali ,  Hemal Khatri ,  Ju Wang

IPC分类号： G06F12/16

CPC分类号： G06F11/2094 ,  G06F3/0623 ,  G06F3/0647 ,  G06F3/067 ,  G06F11/2097

摘要： Embodiments of the present invention relate to invoking and managing migration operations applied to partitions within a distributed computing environment, where each partition represents a key range of data for a storage account. The partitions affected by the migration operations are source partitions hosted on a primary storage stamp and/or destination partitions hosted on a secondary storage stamp, where the primary and secondary storage stamps are equipped to replicate the storage account's data therebetween upon initiating a migration. Upon substantial completion of a bootstrapping phase of replication, one migration operation includes designating the secondary storage stamp as a new primary storage stamp such that the destination partitions commence processing client requests, sending resultant transactions to the source partitions, and providing read and write access thereto. Another migration operation includes designating the primary storage stamp as a new secondary storage stamp such that the source partitions commence replaying the transactions.

摘要翻译： 本发明的实施例涉及调用和管理应用于分布式计算环境中的分区的迁移操作，其中每个分区表示存储帐户的数据的关键范围。 由迁移操作影响的分区是托管在辅助存储戳记上托管的主存储戳记和/或目标分区上的源分区，主存储戳记和辅助存储戳记在启动迁移时可以复制存储帐户的数据。 在大量完成复制的引导阶段之后，一个迁移操作包括将辅助存储标记指定为新的主存储标记，使得目的地分区开始处理客户端请求，将结果事务发送到源分区，并提供对其的读取和写入访问 。 另一个迁移操作包括将主存储戳记指定为新的辅助存储戳，使得源分区开始重播事务。

公开(公告)号：US08918362B2

公开(公告)日：2014-12-23

申请号：US13113718

申请日：2011-05-23

申请人： Bradley Gene Calder ,  Niranjan Nilakantan ,  Shashwat Srivastav ,  Jiesheng Wu ,  Abdul Rafay Abbasi ,  Shane Mainali ,  Padmanabha Chakravarthy Uddaraju

发明人： Bradley Gene Calder ,  Niranjan Nilakantan ,  Shashwat Srivastav ,  Jiesheng Wu ,  Abdul Rafay Abbasi ,  Shane Mainali ,  Padmanabha Chakravarthy Uddaraju

IPC分类号： G06F17/00 ,  G06F7/00 ,  G06F17/30

CPC分类号： G06F17/30575

摘要： Embodiments of the present invention relate to systems, methods, and computer storage media for replicating data in a distributed computing environment utilizing a combination of replication methodologies. A full-object replication may be utilized to replicate a full state of an object from a primary data store to a secondary data store. A checkpoint created after initiating the full-object replication may be parsed to identify changes to the object that have been entered since initiating the full-object replication. This replication process is referred to as a delta-checkpoint replication methodology. Additionally, in an embodiment, a log-based replication methodology may be utilized. The log-based replication may communicate data from a log of the primary data store to the secondary data store. It is also contemplated in an exemplary embodiment that when the log-based replication fails to maintain a throughput threshold, one of the other replication methodologies may be initiated, at least temporarily.

摘要翻译： 本发明的实施例涉及利用复制方法的组合在分布式计算环境中复制数据的系统，方法和计算机存储介质。 可以使用全对象复制来将对象的完整状态从主数据存储复制到辅助数据存储。 启动全对象复制后创建的检查点可能被解析，以识别自启动全对象复制以来已输入的对象的更改。 该复制过程被称为delta检查点复制方法。 另外，在一个实施例中，可以使用基于日志的复制方法。 基于日志的复制可以将数据从主数据存储的日志传送到辅助数据存储。 在示例性实施例中也可以想到，当基于日志的复制不能维持吞吐量阈值时，可以至少暂时启动其他复制方法之一。

公开(公告)号：US20120303593A1

公开(公告)日：2012-11-29

申请号：US13113634

申请日：2011-05-23

申请人： Bradley Gene Calder ,  Niranjan Nilakantan ,  Shashwat Srivastav ,  Jiesheng Wu ,  Abdul Rafay Abbasi ,  Shane Mainali ,  Padmanabha Chakravarthy Uddaraju

发明人： Bradley Gene Calder ,  Niranjan Nilakantan ,  Shashwat Srivastav ,  Jiesheng Wu ,  Abdul Rafay Abbasi ,  Shane Mainali ,  Padmanabha Chakravarthy Uddaraju

IPC分类号： G06F17/30

CPC分类号： G06F11/1004 ,  G06F17/30575

摘要： Presented herein are methods of continuously verifying data and repairing errors introduced during replication. In a particular embodiment, a primary data store sends out information sufficient to create a checkpoint together with a checksum for the data being verified at that checkpoint. At the secondary data store, a checkpoint is created in accordance with the checkpointing information, and a checksum is calculated over the indicated data at the created checkpoint. If the calculated checksum disagrees with the received checksum, additional checksums are calculated over subranges of the indicated data and compared with corresponding checksums over the data at the primary data store. The checksums at the primary data store may be requested from the primary data store or calculated locally based on the received overall checksum. Once an erroneous entry is identified, it can then be re-replicated from the primary data store to restore data consistency.

摘要翻译： 这里提出的是连续验证数据和修复复制期间引入的错误的方法。 在特定实施例中，主数据存储器发出足以创建检查点的信息以及在该检查点处正在验证的数据的校验和。 在次级数据存储中，根据检查点信息创建检查点，并且在创建的检查点处对指示的数据计算校验和。 如果计算的校验和与接收到的校验和不一致，则在指定数据的子范围内计算额外的校验和，并与主数据存储器上的数据相对应的校验和进行比较。 可以从主数据存储器请求主数据存储器上的校验和，或者基于接收到的总体校验和在本地计算。 一旦识别出错误的条目，它就可以从主数据存储重新复制，以恢复数据的一致性。

公开(公告)号：US20120303581A1

公开(公告)日：2012-11-29

申请号：US13113718

申请日：2011-05-23

申请人： Bradley Gene Calder ,  Niranjan Nilakantan ,  Shashwat Srivastav ,  Jiesheng Wu ,  Abdul Rafay Abbasi ,  Shane Mainali ,  Padmanabha Chakravarthy Uddaraju

发明人： Bradley Gene Calder ,  Niranjan Nilakantan ,  Shashwat Srivastav ,  Jiesheng Wu ,  Abdul Rafay Abbasi ,  Shane Mainali ,  Padmanabha Chakravarthy Uddaraju

IPC分类号： G06F17/30

CPC分类号： G06F17/30575

摘要： Embodiments of the present invention relate to systems, methods, and computer storage media for replicating data in a distributed computing environment utilizing a combination of replication methodologies. A full-object replication may be utilized to replicate a full state of an object from a primary data store to a secondary data store. A checkpoint created after initiating the full-object replication may be parsed to identify changes to the object that have been entered since initiating the full-object replication. This replication process is referred to as a delta-checkpoint replication methodology. Additionally, in an embodiment, a log-based replication methodology may be utilized. The log-based replication may communicate data from a log of the primary data store to the secondary data store. It is also contemplated in an exemplary embodiment that when the log-based replication fails to maintain a throughput threshold, one of the other replication methodologies may be initiated, at least temporarily.

摘要翻译： 本发明的实施例涉及利用复制方法的组合在分布式计算环境中复制数据的系统，方法和计算机存储介质。 可以使用全对象复制来将对象的完整状态从主数据存储复制到辅助数据存储。 启动全对象复制后创建的检查点可能被解析，以识别自启动全对象复制以来已输入的对象的更改。 该复制过程被称为delta检查点复制方法。 另外，在一个实施例中，可以使用基于日志的复制方法。 基于日志的复制可以将数据从主数据存储的日志传送到辅助数据存储。 在示例性实施例中也可以想到，当基于日志的复制不能维持吞吐量阈值时，可以至少暂时启动其他复制方法之一。

公开(公告)号：US20120303576A1

公开(公告)日：2012-11-29

申请号：US13113702

申请日：2011-05-23

申请人： Bradley Gene Calder ,  Niranjan Nilakantan ,  Shashwat Srivastav ,  Jiesheng Wu ,  Abdul Rafay Abbasi ,  Shane Mainali ,  Padmanabha Chakravarthy Uddaraju

发明人： Bradley Gene Calder ,  Niranjan Nilakantan ,  Shashwat Srivastav ,  Jiesheng Wu ,  Abdul Rafay Abbasi ,  Shane Mainali ,  Padmanabha Chakravarthy Uddaraju

IPC分类号： G06F7/00

CPC分类号： G06F17/30581 ,  G06F11/2074 ,  G06F11/2076 ,  G06F17/30362 ,  G06F17/30377 ,  G06F17/30575 ,  G06F2201/855

摘要： Embodiments of the present invention relate to synchronously replicating data in a distributed computing environment. To achieve synchronous replication both an eventual consistency approach and a strong consistency approach are contemplated. Received data may be written to a log of a primary data store for eventual committal. The data may then be annotated with a record, such as a unique identifier, which facilitates the replay of the data at a secondary data store. Upon receiving an acknowledgment that the secondary data store has written the data to a log, the primary data store may commit the data and communicate an acknowledgment of success back to the client. In a strong consistency approach, the primary data store may wait to send an acknowledgement of success to the client until it receives an acknowledgment that the secondary has not only written, but also committed, the data.

摘要翻译： 本发明的实施例涉及在分布式计算环境中同步复制数据。 为了实现同步复制，可以考虑最终一致性方法和强一致性方法。 收到的数据可能会写入主数据存储的日志，以便最终提交。 然后可以用诸如唯一标识符的记录来注释数据，这有助于辅助数据存储器处的数据的重放。 在收到辅助数据存储已经将数据写入日志的确认之后，主数据存储可以提交数据并将成功确认传回到客户端。 在强一致性方法中，主数据存储可以等待向客户端发送成功确认，直到其接收到确认次要不仅写入但已经提交的数据。

公开(公告)号：US08495036B2

公开(公告)日：2013-07-23

申请号：US12257786

申请日：2008-10-24

申请人： Bradley Gene Calder ,  Ju Wang ,  Xinran Wu ,  Niranjan Nilakantan ,  Deepali Bhardwaj ,  Shashwat Srivastav ,  Alexander Felsöbüki Nagy

发明人： Bradley Gene Calder ,  Ju Wang ,  Xinran Wu ,  Niranjan Nilakantan ,  Deepali Bhardwaj ,  Shashwat Srivastav ,  Alexander Felsöbüki Nagy

IPC分类号： G06F7/00 ,  G06F17/00

CPC分类号： G06F17/30303 ,  G06F17/30011 ,  G06F17/30017

摘要： Embodiments of the present invention relate to systems, methods and computer storage media for facilitating the structured storage of binary large objects (Blobs) to be accessed by an application program being executed by a computing device. Generally, the manipulation of Blobs in a structured storage system includes receiving a request for a Blob, which may be located by way of a Blob pointer. The Blob pointer allows for the data, such as properties, of the Blob to be identified and located. Expired properties are garbage collected as a manipulation of the Blob data within a structured storage system. In an embodiment, the Blob is identified by a key that is utilized within a primary structured index to located the requested Blob. In another embodiment, the requested Blob is located utilizing a secondary hash index. In an additional embodiment, the Blob is locate utilizing a file table.

摘要翻译： 本发明的实施例涉及用于促进由计算设备执行的应用程序访问的二进制大对象（Blob）的结构化存储的系统，方法和计算机存储介质。 通常，结构化存储系统中Blob的操作包括接收对可以通过Blob指针定位的Blob的请求。 Blob指针允许识别和定位Blob的数据，例如属性。 过期属性作为在结构化存储系统中操纵Blob数据而被垃圾回收。 在一个实施例中，Blob由在主要结构化索引中使用的密钥来标识以定位所请求的Blob。 在另一个实施例中，使用辅助散列索引定位所请求的Blob。 在另外的实施例中，使用文件表定位Blob。

公开(公告)号：US08620884B2

公开(公告)日：2013-12-31

申请号：US12257747

申请日：2008-10-24

申请人： Bradley Gene Calder ,  Ju Wang ,  Xinran Wu ,  Niranjan Nilakantan ,  Deepali Bhardwaj ,  Shashwat Srivastav ,  Alexander Felsöbüki Nagy

发明人： Bradley Gene Calder ,  Ju Wang ,  Xinran Wu ,  Niranjan Nilakantan ,  Deepali Bhardwaj ,  Shashwat Srivastav ,  Alexander Felsöbüki Nagy

IPC分类号： G06F7/00

CPC分类号： G06F17/3033 ,  G06F17/30501

摘要： Embodiments of the present invention relate to systems, methods and computer storage media for facilitating the structured storage of binary large objects (Blobs) to be accessed by an application program being executed by a computing device. Generally, the structured storage of Blobs includes a primary structured storage index for indexing Blobs, a secondary hash index that is integrated into the structured storage system, a Blob log stream, and a Blob data stream for storing blocks that include the Blob data. In an embodiment, a block is created and written to a Blob store along with a block list. The block list facilitates the locating of one or more blocks that store the Blob data. In this embodiment, a primary structured storage index and a secondary hash index are updated to facilitate efficient access of the Blob in a structured storage system.

摘要翻译： 本发明的实施例涉及用于促进由计算设备执行的应用程序访问的二进制大对象（Blob）的结构化存储的系统，方法和计算机存储介质。 通常，Blob的结构化存储包括用于索引Blob的主要结构化存储索引，集成到结构化存储系统中的辅助散列索引，Blob日志流和用于存储包含Blob数据的块的Blob数据流。 在一个实施例中，块被创建并且被写入到Blob存储以及块列表。 块列表便于定位存储Blob数据的一个或多个块。 在该实施例中，更新主结构化存储索引和辅助散列索引以便于结构化存储系统中Blob的有效访问。

公开(公告)号：US09996572B2

公开(公告)日：2018-06-12

申请号：US12258050

申请日：2008-10-24

申请人： Bradley Gene Calder ,  Ju Wang ,  Arild E. Skjolsvold ,  Shashwat Srivastav ,  Niranjan Nilakantan ,  Deepali Bhardwaj

发明人： Bradley Gene Calder ,  Ju Wang ,  Arild E. Skjolsvold ,  Shashwat Srivastav ,  Niranjan Nilakantan ,  Deepali Bhardwaj

IPC分类号： G06F7/00 ,  G06F17/30 ,  G06F3/06

CPC分类号： G06F17/30339 ,  G06F3/0605 ,  G06F3/0644 ,  G06F3/067 ,  G06F2201/86

摘要： Partition management for a scalable, structured storage system is provided. The storage system provides storage represented by one or more tables, each of which includes rows that represent data entities. A table is partitioned into a number of partitions, each partition including a contiguous range of rows. The partitions are served by table servers and managed by a table master. Load distribution information for the table servers and partitions is tracked, and the table master determines to split and/or merge partitions based on the load distribution information.

公开(公告)号：US08769134B2

公开(公告)日：2014-07-01

申请号：US13471706

申请日：2012-05-15

申请人： Bradley Gene Calder ,  Niranjan Nilakantan ,  Padmanabha Chakravarthy Uddaraju

发明人： Bradley Gene Calder ,  Niranjan Nilakantan ,  Padmanabha Chakravarthy Uddaraju

IPC分类号： G06F15/16 ,  G06F15/173 ,  G06F3/00

CPC分类号： G06F9/546

摘要： A cloud computing platform contains a structured storage subsystem the provides scalable queues. The cloud computing platform monitors message throughput for the scalable queues and automatically increases or decreases subqueues that provide the operational functionality for each scalable queue. A visibility start time and cloud computing platform time are maintained for each message to provide an approximate first-in-first-out order for messages within each subqueue. A message in a subqueue may be available for processing when the current cloud computing time is greater than the visibility start of the message.

摘要翻译： 云计算平台包含一个结构化的存储子系统，提供可扩展的队列。 云计算平台监视可扩展队列的消息吞吐量，并自动增加或减少为每个可扩展队列提供操作功能的子队列。 为每个消息维护可见性开始时间和云计算平台时间，以便为每个子队列中的消息提供大致的先进先出顺序。 当当前云计算时间大于消息的可见性开始时，子队列中的消息可能可用于处理。