-
公开(公告)号:US20160077746A1
公开(公告)日:2016-03-17
申请号:US14484820
申请日:2014-09-12
Applicant: NetApp, Inc.
Inventor: John Muth , Edward D. McClanahan , Dhaval Patel , Manish Swaminathan
IPC: G06F3/06
CPC classification number: G06F3/0652 , G06F3/0608 , G06F3/061 , G06F3/0644 , G06F3/0673 , G06F3/0688 , G06F2212/7205
Abstract: An optimized segment cleaning technique is configured to efficiently clean one or more selected portions or segments of a storage array coupled to one or more nodes of a cluster. A bottom-up approach of the segment cleaning technique is configured to read all blocks of a segment to be cleaned (i.e., an “old” segment) to locate extents stored on the SSDs of the old segment and examine extent metadata to determine whether the extents are valid and, if so, relocate the valid extents to a segment being written (i.e., a “new” segment). A top-down approach of the segment cleaning technique obviates reading of the blocks of the old segment to locate the extents and, instead, examines the extent metadata to determine the valid extents of the old segment. A hybrid approach may extend the top-down approach to include only full stripe read operations needed for relocation and reconstruction of blocks as well as retrieval of valid extents from the stripes, while also avoiding any unnecessary read operations of the bottom-down approach.
Abstract translation: 经优化的段清洁技术被配置为有效地清洁耦合到群集的一个或多个节点的存储阵列的一个或多个选定部分或段。 分段清洁技术的自下而上的方法被配置为读取待清理的段的所有块(即,“旧”段))以定位存储在旧段的SSD上的区段,并检查扩展元数据以确定是否 范围是有效的,如果是,将有效范围重定位到正在写入的段(即“新”段)。 段清除技术的自上而下的方法避免了读取旧段的块以定位范围,而是检查元数据的大小以确定旧段的有效范围。 混合方法可以扩展自上而下的方法,以仅包括块的重定位和重建所需的全条带读取操作以及从条带检索有效范围,同时还避免了自下而上的方法的任何不必要的读操作。
-
公开(公告)号:US10762070B2
公开(公告)日:2020-09-01
申请号:US15895593
申请日:2018-02-13
Applicant: NetApp, Inc.
Inventor: Manish Swaminathan , Dhaval Patel , Edward D. McClanahan , Jeffrey S. Kimmel
IPC: G06F16/22 , G06F12/02 , G06F16/84 , G06F16/182
Abstract: A technique reduces an amount of metadata stored in a memory of a node in a cluster. An extent store layer of a storage input/output (I/O) stack executing on the node stores key-value pairs in a plurality of data structures, e.g., cuckoo hash tables, resident in the memory. The cuckoo hash table embodies metadata that describes an extent and, as such, may be organized to associate a location on disk with a value that identifies the location on disk. The value may be embodied as a locator that includes a reference count used to support deduplication functionality of the extent store layer with respect to the extent. The reference count is divided into two portions: a delta count portion stored in memory for each slot of the hash table and an overflow count portion stored on disk in a header of each extent. One bit of the delta count portion is reserved as an overflow bit that indicates whether the in-memory reference count has overflowed. Another bit of the delta count portion is reserved as a sign bit that indicates whether the value of the remaining delta count portion, which stores the “delta” of the reference count, is positive or negative. Overflow updates to the overflow count portion on disk are postponed until all of the bits of the delta count portion are consumed as negative/positive transitions.
-
公开(公告)号:US10210082B2
公开(公告)日:2019-02-19
申请号:US15581800
申请日:2017-04-28
Applicant: NetApp, Inc.
Inventor: Dhaval Patel , Manish Swaminathan , Edward D. McClanahan , John Muth
Abstract: A rate matching technique may be configured to adjust a rate of cleaning of one or more selected segments of the storage array to accommodate a variable rate of incoming workload processed by a storage input/output (I/O) stack executing on one or more nodes of a cluster. An extent store layer of the storage I/O stack may clean a segment in accordance with segment cleaning which, illustratively, may be embodied as a segment cleaning process. The rate matching technique may be implemented as a feedback control mechanism configured to adjust the segment cleaning process based on the incoming workload. Components of the feedback control mechanism may include one or more weight schedulers and various accounting data structures, e.g., counters, configured to track the progress of segment cleaning and free space usage. The counters may also be used to balance the rates of segment cleaning and incoming I/O workload, which may change depending upon an incoming I/O rate. When the incoming I/O rate changes, the rate of segment cleaning may be adjusted accordingly to ensure that rates are substantially balanced.
-
公开(公告)号:US09934264B2
公开(公告)日:2018-04-03
申请号:US14728482
申请日:2015-06-02
Applicant: NetApp, Inc.
Inventor: Manish Swaminathan , Dhaval Patel , Edward D. McClanahan , Jeffrey S. Kimmel
CPC classification number: G06F17/3033 , G06F12/0246 , G06F12/0261 , G06F17/30194 , G06F17/30914
Abstract: A technique reduces an amount of metadata stored in a memory of a node in a cluster. An extent store layer of a storage input/output (I/O) stack executing on the node stores key-value pairs in a plurality of data structures, e.g., cuckoo hash tables, resident in the memory. The cuckoo hash table embodies metadata that describes an extent and, as such, may be organized to associate a location on disk with a value that identifies the location on disk. The value may be embodied as a locator that includes a reference count used to support deduplication functionality of the extent store layer with respect to the extent. The reference count is divided into two portions: a delta count portion stored in memory for each slot of the hash table and an overflow count portion stored on disk in a header of each extent. One bit of the delta count portion is reserved as an overflow bit that indicates whether the in-memory reference count has overflowed. Another bit of the delta count portion is reserved as a sign bit that indicates whether the value of the remaining delta count portion, which stores the “delta” of the reference count, is positive or negative. Overflow updates to the overflow count portion on disk are postponed until all of the bits of the delta count portion are consumed as negative/positive transitions.
-
公开(公告)号:US09671960B2
公开(公告)日:2017-06-06
申请号:US14484565
申请日:2014-09-12
Applicant: NetApp, Inc.
Inventor: Dhaval Patel , Manish Swaminathan , Edward D. McClanahan , John Muth
CPC classification number: G06F12/0253 , G06F3/0608 , G06F3/061 , G06F3/0619 , G06F3/0641 , G06F3/0656 , G06F3/0688 , G06F12/0246 , G06F2212/1016 , G06F2212/1024 , G06F2212/1044 , G06F2212/702 , G06F2212/7201 , G06F2212/7205
Abstract: A rate matching technique may be configured to adjust a rate of cleaning of one or more selected segments of the storage array to accommodate a variable rate of incoming workload processed by a storage input/output (I/O) stack executing on one or more nodes of a cluster. An extent store layer of the storage I/O stack may clean a segment in accordance with segment cleaning which, illustratively, may be embodied as a segment cleaning process. The rate matching technique may be implemented as a feedback control mechanism configured to adjust the segment cleaning process based on the incoming workload. Components of the feedback control mechanism may include one or more weight schedulers and various accounting data structures, e.g., counters, configured to track the progress of segment cleaning and free space usage. The counters may also be used to balance the rates of segment cleaning and incoming I/O workload, which may change depending upon an incoming I/O rate. When the incoming I/O rate changes, the rate of segment cleaning may be adjusted accordingly to ensure that rates are substantially balanced.
-
-
-
-