Batch building for deep learning training workloads

    公开(公告)号:US11455168B1

    公开(公告)日:2022-09-27

    申请号:US16747854

    申请日:2020-01-21

    Abstract: Batch building for artificial intelligence workflows, including: issuing, responsive to a request for a batch of data objects, requests to a data repository for multiple data objects stored among one or more directories; selecting, in accordance with a batch building policy, a subset of data objects based on one or more responses to the requests; and providing, to the artificial intelligence workflow, a batch of data objects that includes the subset of data objects.

    Action determination based on redundancy level

    公开(公告)号:US11138082B2

    公开(公告)日:2021-10-05

    申请号:US16576084

    申请日:2019-09-19

    Abstract: A plurality of storage nodes is provided. Each of the plurality of storage nodes includes nonvolatile solid-state memory for user data storage. The plurality of storage nodes is configured to distribute the user data and metadata associated with the user data throughout the plurality of storage nodes such that the plurality of storage nodes maintain the ability to read the user data, using erasure coding, despite a loss of two of the plurality of storage nodes. The plurality of storage nodes is configured to initiate an action based on the redundant copies of the metadata, responsive to achieving a level of redundancy for the redundant copies of the metadata. A method for accessing user data in a plurality of storage nodes having nonvolatile solid-state memory is also provided.

    Direct Memory Access Data Format
    25.
    发明申请

    公开(公告)号:US20210073149A1

    公开(公告)日:2021-03-11

    申请号:US16953171

    申请日:2020-11-19

    Abstract: A method of applying a data format in a direct memory access transfer is provided. The method includes distributing user data throughout a plurality of storage nodes through erasure coding, wherein the plurality of storage nodes are housed within a single chassis that couples the storage nodes as a cluster, each of the plurality of storage nodes having nonvolatile solid-state memory for user data storage. The method includes reading a self-describing data portion from a first memory of the nonvolatile solid-state memory and extracting a destination from the self-describing data portion. The method includes writing data, from the self-describing data portion, to a second memory of the nonvolatile solid-state memory according to the destination.

    Direct memory access data format
    26.
    发明授权

    公开(公告)号:US10853285B2

    公开(公告)日:2020-12-01

    申请号:US16267163

    申请日:2019-02-04

    Abstract: A method of applying a data format in a direct memory access transfer is provided. The method includes distributing user data throughout a plurality of storage nodes through erasure coding, wherein the plurality of storage nodes are housed within a single chassis that couples the storage nodes as a cluster, each of the plurality of storage nodes having nonvolatile solid-state memory for user data storage. The method includes reading a self-describing data portion from a first memory of the nonvolatile solid-state memory and extracting a destination from the self-describing data portion. The method includes writing data, from the self-describing data portion, to a second memory of the nonvolatile solid-state memory according to the destination.

    Direct memory access data movement
    27.
    发明授权

    公开(公告)号:US10198380B1

    公开(公告)日:2019-02-05

    申请号:US15686956

    申请日:2017-08-25

    Abstract: A method of applying a data format in a direct memory access transfer is provided. The method includes distributing user data throughout a plurality of storage nodes through erasure coding, wherein the plurality of storage nodes are housed within a single chassis that couples the storage nodes as a cluster, each of the plurality of storage nodes having nonvolatile solid-state memory for user data storage. The method includes reading a self-describing data portion from a first memory of the nonvolatile solid-state memory and extracting a destination from the self-describing data portion. The method includes writing data, from the self-describing data portion, to a second memory of the nonvolatile solid-state memory according to the destination.

    OPTIMIZE DATA PROTECTION LAYOUTS BASED ON DISTRIBUTED FLASH WEAR LEVELING

    公开(公告)号:US20180024877A1

    公开(公告)日:2018-01-25

    申请号:US15592069

    申请日:2017-05-10

    Abstract: A method for storing data in a storage system having solid-state memory is provided. The method includes determining portions of the solid-state memory that have a faster access rate and portions of the solid-state memory that have a slower access rate, relative to each other or to a threshold. The method includes writing data bits of erasure coded data to the portions of the solid-state memory having the faster access rate, and writing one or more parity bits of the erasure coded data to the portions of the solid-state memory having the slower access rate. A storage system is also provided.

    Scheduling policy for queues in a non-volatile solid-state storage
    30.
    发明授权
    Scheduling policy for queues in a non-volatile solid-state storage 有权
    在非易失性固态存储中排队的调度策略

    公开(公告)号:US09501244B2

    公开(公告)日:2016-11-22

    申请号:US14523719

    申请日:2014-10-24

    Abstract: A method of applying scheduling policies is provided. The method includes distributing user data throughout a plurality of storage nodes through erasure coding, wherein the plurality of storage nodes are housed within a single chassis coupling the storage nodes as a cluster. The method includes receiving operations relating to a non-volatile memory of one of the plurality of storage nodes into a plurality of operation queues. The method includes evaluating each of the operations in the plurality of operation queues as to benefit to the non-volatile solid-state storage according to a plurality of policies. For each channel of a plurality of channels coupling the operation queues to the non-volatile memory, the method includes iterating a selection and an execution of a next operation from the plurality of operation queues, with each next operation having a greater benefit than at least a subset of operations remaining in the operation queues.

    Abstract translation: 提供了一种应用调度策略的方法。 该方法包括通过擦除编码在多个存储节点中分配用户数据,其中多个存储节点被容纳在将存储节点耦合为一个簇的单个机架中。 该方法包括将与多个存储节点之一的非易失性存储器相关的操作接收到多个操作队列中。 该方法包括评估多个操作队列中的每个操作,以便根据多个策略使非易失性固态存储器受益。 对于将操作队列耦合到非易失性存储器的多个通道中的每个通道,该方法包括从多个操作队列中迭代选择和执行下一操作,其中每个下一操作具有比至少 操作队列中剩余的一组操作。

Patent Agency Ranking