Multi-level lookup architecture to facilitate failure recovery
    1.
    发明授权
    Multi-level lookup architecture to facilitate failure recovery 有权
    多级查找架构,方便故障恢复

    公开(公告)号:US09405643B2

    公开(公告)日:2016-08-02

    申请号:US14091119

    申请日:2013-11-26

    Applicant: Dropbox, Inc.

    CPC classification number: G06F11/2094 G06F17/30212 G06F17/3033

    Abstract: The disclosed embodiments relate to a data storage system that facilitates efficiently recovering from storage device failures. Upon receiving a request to retrieve a data block from the data storage system, the system uses a hash that identifies the data block to look up a bucket and an associated cell containing the data block. Note that the bucket aggregates a large number of data blocks and is located in the associated cell that comprises a set of object storage devices (OSDs). Within the cell, the system uses the bucket to look up an OSD that contains the bucket in a local bucket database (BDB) for the cell. Within the OSD, the system uses the bucket and the hash to look up an offset and a length for the data block in a write-ahead log that stores data blocks for the bucket. Finally, the system returns the data block from the determined offset.

    Abstract translation: 所公开的实施例涉及有助于从存储设备故障有效地恢复的数据存储系统。 在接收到从数据存储系统检索数据块的请求时,系统使用标识数据块的散列来查找存储桶和包含该数据块的关联单元。 注意,桶聚合大量的数据块并且位于包括一组对象存储设备(OSD)的关联小区中)。 在单元格内,系统使用存储桶在单元的本地存储桶数据库(BDB)中查找包含存储桶的OSD。 在OSD中,系统使用存储桶和散列来在存储数据块的预写日志中查找数据块的偏移量和长度。 最后,系统从确定的偏移量返回数据块。

    DISTRIBUTED TECHNIQUE FOR ALLOCATING LONG-LIVED JOBS AMONG WORKER PROCESSES
    2.
    发明申请
    DISTRIBUTED TECHNIQUE FOR ALLOCATING LONG-LIVED JOBS AMONG WORKER PROCESSES 审中-公开
    在工作流程中分配长期工作的分配技术

    公开(公告)号:US20160147569A1

    公开(公告)日:2016-05-26

    申请号:US14555101

    申请日:2014-11-26

    Applicant: Dropbox, Inc.

    CPC classification number: G06F3/067 G06F9/5027 G06F17/30312

    Abstract: A distributed computing system that executes a set of long-lived jobs is described. During operation, each worker process performs the following operations. First, the worker process identifies a set of jobs to be executed and a set of worker processes that can execute the set of jobs. Next, the worker process sorts the set of worker processes based on unique identifiers for the worker processes. Then, the worker process assigns jobs to each worker process in the set of worker processes, wherein approximately the same number of jobs is assigned to each worker process, and jobs are assigned to the worker processes in sorted order. While assigning jobs, the worker process uses an identifier for each worker process to seed a pseudorandom number generator, and then uses the pseudorandom number generator to select jobs for each worker process to execute.

    Abstract translation: 描述执行一组长寿命作业的分布式计算系统。 在操作期间,每个工作进程执行以下操作。 首先,工作进程标识要执行的一组作业和一组可以执行作业的工作进程。 接下来,工作进程根据工作进程的唯一标识符对工作进程集进行排序。 然后,工作进程将作业分配给一组工作进程中的每个工作进程,其中大约相同数量的作业分配给每个工作进程,作业按排序顺序分配给工作进程。 在分配作业时,工作进程使用每个工作进程的标识符来种子伪随机数生成器,然后使用伪随机数生成器为每个要执行的工作进程选择作业。

    USING SCRATCH EXTENTS TO FACILITATE COPYING OPERATIONS IN AN APPEND-ONLY STORAGE SYSTEM
    3.
    发明申请
    USING SCRATCH EXTENTS TO FACILITATE COPYING OPERATIONS IN AN APPEND-ONLY STORAGE SYSTEM 审中-公开
    使用剪切存在可以在附加的存储系统中实现复制操作

    公开(公告)号:US20160110109A1

    公开(公告)日:2016-04-21

    申请号:US14519907

    申请日:2014-10-21

    Applicant: Dropbox, Inc.

    Abstract: An append-only data storage system is described that stores sets of data blocks in extents that are located in storage devices. During operation of the system, upon receiving a request to copy an extent from a source storage device to a destination storage device, the system creates a scratch extent on the destination storage device, and associates the scratch extent with a private identifier, whereby the scratch extent can only be accessed through the private identifier. The system uses the private identifier to perform a copying operation that copies the extent from the source storage device to the scratch extent on the destination storage device. After the copying operation is complete and the scratch extent is closed, the system associates the scratch extent with a public identifier, whereby the copy of the extent on the destination storage device becomes publically accessible to other entities in the data storage system.

    Abstract translation: 描述了仅附加数据存储系统,其存储位于存储设备中的扩展区中的数据块集合。 在系统操作期间,当接收到从源存储设备向目标存储设备复制范围的请求时,系统在目的地存储设备上创建临时区段,并将临时区段与私有标识符相关联,由此划伤 范围只能通过私有标识符访问。 该系统使用专用标识符执行复制操作,该操作将来自源存储设备的扩展区域复制到目标存储设备上的临时区域。 在复制操作完成并且临时区段关闭之后,系统将临时区段与公共标识符相关联,由此目的地存储设备上的区段的副本可以公开地访问数据存储系统中的其他实体。

    Erasure-coding extents in an append-only storage system

    公开(公告)号:US09619322B2

    公开(公告)日:2017-04-11

    申请号:US14540610

    申请日:2014-11-13

    Applicant: Dropbox, Inc.

    CPC classification number: G06F11/1048 H03M13/1515

    Abstract: A data storage system stores sets of data blocks in extents located on storage devices. During operation, the system performs an erasure-coding operation by obtaining a set of source extents, wherein each source extent is stored on a different machine in the data storage system. The system also selects a set of destination machines for storing destination extents, wherein each destination extent is stored on a different destination machine. Next, the system performs the erasure-coding operation by retrieving data from the set of source extents, performing the erasure-coding operation on the retrieved data to produce erasure-coded data, and then writing the erasure-coded data to the set of destination extents on the set of destination machines. Finally, after the erasure-coding operation is complete, the system commits results of the erasure-coding operation to enable the set of destination extents to be accessed in place of the set of source extents.

    SYNCHRONIZING COPIES OF AN EXTENT IN AN APPEND-ONLY STORAGE SYSTEM
    7.
    发明申请
    SYNCHRONIZING COPIES OF AN EXTENT IN AN APPEND-ONLY STORAGE SYSTEM 有权
    在同一存储系统中同步复制副本

    公开(公告)号:US20160092491A1

    公开(公告)日:2016-03-31

    申请号:US14497212

    申请日:2014-09-25

    Applicant: Dropbox, Inc.

    Abstract: A append-only data storage system that stores sets of data blocks in extents that are located in storage devices. When an extent becomes full, the system changes the extent from an open state, wherein data can be appended to the extent, to a closed state, wherein data cannot be appended to the extent. This change involves performing a synchronization operation by: obtaining a list of data blocks in the extent from each storage device that has a copy of the extent; forming a union of the lists; looking up data blocks from the union in a database that maps data blocks to storage devices and extents to determine which data blocks belong in the extent; and if a copy of the extent is missing data blocks that belong in the extent, performing a remedial action before changing the extent from the open state to the closed state.

    Abstract translation: 一种仅追加数据存储系统,用于存储位于存储设备中的扩展区中的数据块集。 当范围变满时,系统将从打开状态改变范围,其中可以将数据附加到该范围到关闭状态,其中数据不能被附加到该范围。 该改变涉及通过以下方式来执行同步操作:获得具有范围的副本的每个存储设备的范围内的数据块的列表; 形成名单联盟; 在数据库中从联合查找数据块,将数据块映射到存储设备和扩展区,以确定哪个数据块属于该范围; 并且如果范围的副本缺少属于扩展区的数据块,则在将扩展从更改状态更改为关闭状态之前执行补救措施。

    MULTI-LEVEL LOOKUP ARCHITECTURE TO FACILITATE FAILURE RECOVERY
    8.
    发明申请
    MULTI-LEVEL LOOKUP ARCHITECTURE TO FACILITATE FAILURE RECOVERY 有权
    多层次的建筑架构,以促进失败恢复

    公开(公告)号:US20150149500A1

    公开(公告)日:2015-05-28

    申请号:US14091119

    申请日:2013-11-26

    Applicant: Dropbox, Inc.

    CPC classification number: G06F11/2094 G06F17/30212 G06F17/3033

    Abstract: The disclosed embodiments relate to a data storage system that facilitates efficiently recovering from storage device failures. Upon receiving a request to retrieve a data block from the data storage system, the system uses a hash that identifies the data block to look up a bucket and an associated cell containing the data block. Note that the bucket aggregates a large number of data blocks and is located in the associated cell that comprises a set of object storage devices (OSDs). Within the cell, the system uses the bucket to look up an OSD that contains the bucket in a local bucket database (BDB) for the cell. Within the OSD, the system uses the bucket and the hash to look up an offset and a length for the data block in a write-ahead log that stores data blocks for the bucket. Finally, the system returns the data block from the determined offset.

    Abstract translation: 所公开的实施例涉及有助于从存储设备故障有效地恢复的数据存储系统。 在接收到从数据存储系统检索数据块的请求时,系统使用标识数据块的散列来查找存储桶和包含该数据块的关联单元。 注意,桶聚合大量的数据块并且位于包括一组对象存储设备(OSD)的关联小区中)。 在单元格内,系统使用存储桶在单元的本地存储桶数据库(BDB)中查找包含存储桶的OSD。 在OSD中,系统使用存储桶和散列来在存储数据块的预写日志中查找数据块的偏移量和长度。 最后,系统从确定的偏移量返回数据块。

    Distributed technique for allocating long-lived jobs among worker processes

    公开(公告)号:US10310904B2

    公开(公告)日:2019-06-04

    申请号:US14555101

    申请日:2014-11-26

    Applicant: Dropbox, Inc.

    Abstract: A distributed computing system that executes a set of long-lived jobs is described. During operation, each worker process performs the following operations. First, the worker process identifies a set of jobs to be executed and a set of worker processes that can execute the set of jobs. Next, the worker process sorts the set of worker processes based on unique identifiers for the worker processes. Then, the worker process assigns jobs to each worker process in the set of worker processes, wherein approximately the same number of jobs is assigned to each worker process, and jobs are assigned to the worker processes in sorted order. While assigning jobs, the worker process uses an identifier for each worker process to seed a pseudorandom number generator, and then uses the pseudorandom number generator to select jobs for each worker process to execute.

Patent Agency Ranking