Reconstructing in-memory indices in a distributed data storage system

    公开(公告)号:US10452271B2

    公开(公告)日:2019-10-22

    申请号:US15601598

    申请日:2017-05-22

    Applicant: Dropbox, Inc.

    Abstract: Reconstructing in-memory data block indices in a distributed data storage system where data blocks are stored in extents and the extents are replicated across storage devices. In one aspect, based on a reboot of a storage device and a copy of an extent stored in the storage device being in an open state, appends for data blocks in the copy of the extent stored in the storage device are replayed to reconstruct an in-memory data block index for the copy of the extent. In another aspect, based on a reboot of a storage device and a copy of an extent being in a closed state, a data block index for the copy of the extent is retrieved from non-volatile storage of the storage device and the retrieved data block index stored in memory at the storage device.

    ATOMIC MOVES WITH LAMPORT CLOCKS IN A CONTENT MANAGEMENT SYSTEM

    公开(公告)号:US20190205457A1

    公开(公告)日:2019-07-04

    申请号:US15873693

    申请日:2018-01-17

    Applicant: Dropbox, Inc.

    Abstract: A system can serialize moves and mounts across namespaces based on lamport clocks. In some examples, the system obtains a request to move a content item from a source namespace to a destination namespace. The system processes an incoming move at the destination and an outgoing move at the source. The system processes for the content item a delete at the source and an add at the destination. The system assigns a first clock to the incoming move and a second clock to the outgoing move, the first clock being lower than the second clock. The system assigns a third clock to the delete and a fourth clock to the add, the third clock being higher than the second clock and lower than the fourth clock. The system serializes the incoming and outgoing moves, the delete and the add based on the first, second, third and fourth clocks.

    Using colocation hints to facilitate accessing a distributed data storage system
    26.
    发明授权
    Using colocation hints to facilitate accessing a distributed data storage system 有权
    使用托管提示来方便访问分布式数据存储系统

    公开(公告)号:US09547706B2

    公开(公告)日:2017-01-17

    申请号:US14224786

    申请日:2014-03-25

    Applicant: Dropbox, Inc.

    Inventor: James Cowling

    CPC classification number: G06F17/30581 G06F3/067 G06F17/30194

    Abstract: The disclosed embodiments relate to a system that uses colocation hints to facilitate storing data blocks in a distributed data storage system, which includes a plurality of data centers. During operation, the system receives a write request from a client to write a data block to the distributed data storage system, wherein the write request includes a colocation hint that identifies a colocation group associated with the data block. In response to the write request, the system uses the colocation hint to identify one or more data centers associated with the colocation group. Next, the system writes copies of the data block to the one or more identified data centers. In this way, the system situates copies of data blocks associated with the colocation group in the same data centers when possible.

    Abstract translation: 所公开的实施例涉及使用托槽提示以有助于将数据块存储在包括多个数据中心的分布式数据存储系统中的系统。 在操作期间,系统从客户端接收写请求以将数据块写入分布式数据存储系统,其中写请求包括标识与数据块相关联的托管组的托卡提示。 响应于写入请求,系统使用托管提示来识别与托管组相关联的一个或多个数据中心。 接下来,系统将数据块的副本写入一个或多个识别的数据中心。 以这种方式,如果可能,系统将与托管组相关联的数据块的副本置于同一数据中心。

    USING AN RPC FRAMEWORK TO FACILITATE OUT-OF-BAND DATA TRANSFERS
    27.
    发明申请
    USING AN RPC FRAMEWORK TO FACILITATE OUT-OF-BAND DATA TRANSFERS 有权
    使用RPC框架来简化带外数据传输

    公开(公告)号:US20160112293A1

    公开(公告)日:2016-04-21

    申请号:US14519888

    申请日:2014-10-21

    Applicant: Dropbox, Inc.

    CPC classification number: H04L67/02 G06F9/547 H04L67/1097 H04L67/40

    Abstract: A system that stores sets of data blocks in extents located in storage devices is described. During operation, a receiving device receives, through an RPC framework, a first call asking to transfer an extent from a sending device to the receiving device. In response, the receiving device opens a port for a data connection that operates outside the RPC framework. The receiving device makes a second call, to the sending device through the RPC framework, asking to stream the extent to the port. The receiving device subsequently receives the extent from the sending device through the port and computes a checksum for the extent. The receiving device also receives a return from the second call, including a checksum for the extent computed by the sending device. If the computed checksum matches the received checksum, the receiving device returns the first call to indicate the transfer operation completed successfully.

    Abstract translation: 描述了存储设备中存储数据块集的系统。 在操作期间,接收设备通过RPC框架接收第一个呼叫,要求将发送设备传送到接收设备。 作为响应,接收设备打开在RPC框架之外运行的数据连接的端口。 接收设备通过RPC框架向发送设备发出第二个呼叫,要求将流量传输到端口。 接收设备随后通过端口接收来自发送设备的范围,并计算该范围的校验和。 接收设备还从第二呼叫接收返回,包括由发送设备计算的范围的校验和。 如果计算的校验和与接收到的校验和匹配,则接收设备返回第一个呼叫以指示传输操作成功完成。

    CONSTRUCTING AN INDEX TO FACILITATE ACCESSING A CLOSED EXTENT IN AN APPEND-ONLY STORAGE SYSTEM
    28.
    发明申请
    CONSTRUCTING AN INDEX TO FACILITATE ACCESSING A CLOSED EXTENT IN AN APPEND-ONLY STORAGE SYSTEM 有权
    构建索引以便在附加存储系统中访问封闭的范围

    公开(公告)号:US20160092125A1

    公开(公告)日:2016-03-31

    申请号:US14497200

    申请日:2014-09-25

    Applicant: Dropbox, Inc.

    Abstract: The disclosed embodiments relate to the design of an append-only data storage system that stores sets of data blocks in extents that are located in storage devices in the system. During operation of the system, when an extent becomes full, the system changing the extent from an open state, wherein data can be appended to the extent, to a closed state, wherein data cannot be appended to the extent. Changing the extent from the open state to the closed state includes performing the following operations at one or more storage devices that contain copies of the extent: constructing an index to facilitate accessing data blocks in a copy of the extent contained in the storage device; and appending the index to the copy of the extent in non-volatile storage in the storage device.

    Abstract translation: 所公开的实施例涉及仅在位于系统中的存储设备中的盘区中存储数据块集合的仅追加数据存储系统的设计。 在系统操作期间,当一个档位变满时,该系统将数据从该打开状态改变到一个关闭状态,其中数据不能被附加到该程度。 改变从打开状态到关闭状态的程度包括在包含该范围的副本的一个或多个存储设备处执行以下操作:构建索引以便于访问存储设备中包含的范围的副本中的数据块; 并将索引附加到存储设备中的非易失性存储中的扩展区的副本。

    APPEND-ONLY STORAGE SYSTEM SUPPORTING OPEN AND CLOSED EXTENTS
    29.
    发明申请
    APPEND-ONLY STORAGE SYSTEM SUPPORTING OPEN AND CLOSED EXTENTS 有权
    支持打开和关闭的附加存储系统

    公开(公告)号:US20160092124A1

    公开(公告)日:2016-03-31

    申请号:US14497195

    申请日:2014-09-25

    Applicant: Dropbox, Inc.

    Abstract: The disclosed embodiments relate to the design of an append-only data storage system that stores sets of data blocks in extents that are located in storage devices in the system. During operation of the system, when an extent is in an open state, the system allows data blocks to be appended to the extent, and disallows operations to be performed on the extent that are incompatible with data being concurrently appended to the extent. When the extent becomes full, the system changes the extent from the open state to a closed state. Then, while the extent is in the closed state, the system disallows data blocks to be appended to the extent, and allows operations to be performed on the extent that are incompatible with data being concurrently appended to the extent.

    Abstract translation: 所公开的实施例涉及仅在位于系统中的存储设备中的盘区中存储数据块集合的仅追加数据存储系统的设计。 在系统操作期间,当一个盘区处于打开状态时,该系统允许将数据块附加到该盘区,并且不允许在不兼容同时附加到该盘区的数据的情况下执行操作。 当范围变满时,系统将从打开状态改变到关闭状态。 然后,当该范围处于关闭状态时,系统不允许将数据块附加到该范围,并允许在与同时附加到该范围的数据不兼容的程度上执行操作。

Patent Agency Ranking