SYNCHRONIZING COPIES OF AN EXTENT IN AN APPEND-ONLY STORAGE SYSTEM
    1.
    发明申请
    SYNCHRONIZING COPIES OF AN EXTENT IN AN APPEND-ONLY STORAGE SYSTEM 有权
    在同一存储系统中同步复制副本

    公开(公告)号:US20160092491A1

    公开(公告)日:2016-03-31

    申请号:US14497212

    申请日:2014-09-25

    Applicant: Dropbox, Inc.

    Abstract: A append-only data storage system that stores sets of data blocks in extents that are located in storage devices. When an extent becomes full, the system changes the extent from an open state, wherein data can be appended to the extent, to a closed state, wherein data cannot be appended to the extent. This change involves performing a synchronization operation by: obtaining a list of data blocks in the extent from each storage device that has a copy of the extent; forming a union of the lists; looking up data blocks from the union in a database that maps data blocks to storage devices and extents to determine which data blocks belong in the extent; and if a copy of the extent is missing data blocks that belong in the extent, performing a remedial action before changing the extent from the open state to the closed state.

    Abstract translation: 一种仅追加数据存储系统,用于存储位于存储设备中的扩展区中的数据块集。 当范围变满时,系统将从打开状态改变范围,其中可以将数据附加到该范围到关闭状态,其中数据不能被附加到该范围。 该改变涉及通过以下方式来执行同步操作:获得具有范围的副本的每个存储设备的范围内的数据块的列表; 形成名单联盟; 在数据库中从联合查找数据块,将数据块映射到存储设备和扩展区,以确定哪个数据块属于该范围; 并且如果范围的副本缺少属于扩展区的数据块,则在将扩展从更改状态更改为关闭状态之前执行补救措施。

    Using an RPC framework to facilitate out-of-band data transfers

    公开(公告)号:US09967310B2

    公开(公告)日:2018-05-08

    申请号:US14519888

    申请日:2014-10-21

    Applicant: Dropbox, Inc.

    CPC classification number: H04L67/02 G06F9/547 H04L67/1097 H04L67/40

    Abstract: A system that stores sets of data blocks in extents located in storage devices is described. During operation, a receiving device receives, through an RPC framework, a first call asking to transfer an extent from a sending device to the receiving device. In response, the receiving device opens a port for a data connection that operates outside the RPC framework. The receiving device makes a second call, to the sending device through the RPC framework, asking to stream the extent to the port. The receiving device subsequently receives the extent from the sending device through the port and computes a checksum for the extent. The receiving device also receives a return from the second call, including a checksum for the extent computed by the sending device. If the computed checksum matches the received checksum, the receiving device returns the first call to indicate the transfer operation completed successfully.

    Append-only storage system supporting open and closed extents

    公开(公告)号:US09720607B2

    公开(公告)日:2017-08-01

    申请号:US14497195

    申请日:2014-09-25

    Applicant: Dropbox, Inc.

    Abstract: The disclosed embodiments relate to the design of an append-only data storage system that stores sets of data blocks in extents that are located in storage devices in the system. During operation of the system, when an extent is in an open state, the system allows data blocks to be appended to the extent, and disallows operations to be performed on the extent that are incompatible with data being concurrently appended to the extent. When the extent becomes full, the system changes the extent from the open state to a closed state. Then, while the extent is in the closed state, the system disallows data blocks to be appended to the extent, and allows operations to be performed on the extent that are incompatible with data being concurrently appended to the extent.

    Distributed technique for allocating long-lived jobs among worker processes

    公开(公告)号:US10310904B2

    公开(公告)日:2019-06-04

    申请号:US14555101

    申请日:2014-11-26

    Applicant: Dropbox, Inc.

    Abstract: A distributed computing system that executes a set of long-lived jobs is described. During operation, each worker process performs the following operations. First, the worker process identifies a set of jobs to be executed and a set of worker processes that can execute the set of jobs. Next, the worker process sorts the set of worker processes based on unique identifiers for the worker processes. Then, the worker process assigns jobs to each worker process in the set of worker processes, wherein approximately the same number of jobs is assigned to each worker process, and jobs are assigned to the worker processes in sorted order. While assigning jobs, the worker process uses an identifier for each worker process to seed a pseudorandom number generator, and then uses the pseudorandom number generator to select jobs for each worker process to execute.

    Using scratch extents to facilitate copying operations in an append-only storage system

    公开(公告)号:US10248356B2

    公开(公告)日:2019-04-02

    申请号:US14519907

    申请日:2014-10-21

    Applicant: Dropbox, Inc.

    Abstract: An append-only data storage system is described that stores sets of data blocks in extents that are located in storage devices. During operation of the system, upon receiving a request to copy an extent from a source storage device to a destination storage device, the system creates a scratch extent on the destination storage device, and associates the scratch extent with a private identifier, whereby the scratch extent can only be accessed through the private identifier. The system uses the private identifier to perform a copying operation that copies the extent from the source storage device to the scratch extent on the destination storage device. After the copying operation is complete and the scratch extent is closed, the system associates the scratch extent with a public identifier, whereby the copy of the extent on the destination storage device becomes publically accessible to other entities in the data storage system.

    Reconstructing in-memory indices in a distributed data storage system

    公开(公告)号:US10452271B2

    公开(公告)日:2019-10-22

    申请号:US15601598

    申请日:2017-05-22

    Applicant: Dropbox, Inc.

    Abstract: Reconstructing in-memory data block indices in a distributed data storage system where data blocks are stored in extents and the extents are replicated across storage devices. In one aspect, based on a reboot of a storage device and a copy of an extent stored in the storage device being in an open state, appends for data blocks in the copy of the extent stored in the storage device are replayed to reconstruct an in-memory data block index for the copy of the extent. In another aspect, based on a reboot of a storage device and a copy of an extent being in a closed state, a data block index for the copy of the extent is retrieved from non-volatile storage of the storage device and the retrieved data block index stored in memory at the storage device.

    USING AN RPC FRAMEWORK TO FACILITATE OUT-OF-BAND DATA TRANSFERS
    10.
    发明申请
    USING AN RPC FRAMEWORK TO FACILITATE OUT-OF-BAND DATA TRANSFERS 有权
    使用RPC框架来简化带外数据传输

    公开(公告)号:US20160112293A1

    公开(公告)日:2016-04-21

    申请号:US14519888

    申请日:2014-10-21

    Applicant: Dropbox, Inc.

    CPC classification number: H04L67/02 G06F9/547 H04L67/1097 H04L67/40

    Abstract: A system that stores sets of data blocks in extents located in storage devices is described. During operation, a receiving device receives, through an RPC framework, a first call asking to transfer an extent from a sending device to the receiving device. In response, the receiving device opens a port for a data connection that operates outside the RPC framework. The receiving device makes a second call, to the sending device through the RPC framework, asking to stream the extent to the port. The receiving device subsequently receives the extent from the sending device through the port and computes a checksum for the extent. The receiving device also receives a return from the second call, including a checksum for the extent computed by the sending device. If the computed checksum matches the received checksum, the receiving device returns the first call to indicate the transfer operation completed successfully.

    Abstract translation: 描述了存储设备中存储数据块集的系统。 在操作期间,接收设备通过RPC框架接收第一个呼叫,要求将发送设备传送到接收设备。 作为响应,接收设备打开在RPC框架之外运行的数据连接的端口。 接收设备通过RPC框架向发送设备发出第二个呼叫,要求将流量传输到端口。 接收设备随后通过端口接收来自发送设备的范围,并计算该范围的校验和。 接收设备还从第二呼叫接收返回,包括由发送设备计算的范围的校验和。 如果计算的校验和与接收到的校验和匹配,则接收设备返回第一个呼叫以指示传输操作成功完成。

Patent Agency Ranking