公开(公告)号：US10585627B2

公开(公告)日：2020-03-10

申请号：US15080465

申请日：2016-03-24

Applicant: MICROSOFT TECHNOLOGY LICENSING, LLC

Inventor： Cheng Huang ,  Andrew James Edwards ,  Shane K. Mainali ,  Aaron William Ogus ,  Ioan Oltean ,  Huseyin Simitci ,  Ju Wang ,  Bradley Gene Calder ,  Yikang Xu

IPC: G06F3/00 ,  G06F9/00 ,  G06F3/06 ,  G06F9/445 ,  G06F9/50 ,  G06F9/4401 ,  G06F12/00 ,  G06F12/02

Abstract: In various embodiments, methods and systems for implementing a distributed metadata management system in distributed storage systems are provided. A distributed storage system operates based on data storage resources (e.g., extents and streams). The distributed metadata management system is implemented for extent and stream metadata to facilitate the scalability of metadata processing. The distributed storage system implements extent managers and stream managers that independently manage extent and stream metadata, respectively. The extent managers are associated with an extent table that stores extent metadata. The stream managers are associated with streams that store associations with extents. The distributed metadata management system can also utilize a bootstrap layer that leverages components of a legacy distributed storage system to facilitate distributed management of extent and stream metadata. The bootstrap layer is used to store the extent table as a system table and to persist the state of the stream manager as system streams.

公开(公告)号：US10789223B2

公开(公告)日：2020-09-29

申请号：US15080474

申请日：2016-03-24

Applicant: MICROSOFT TECHNOLOGY LICENSING, LLC

Inventor： Cheng Huang ,  Andrew James Edwards ,  Shane K Mainali ,  Aaron William Ogus ,  Ioan Oltean ,  Huseyin Simitci ,  Ju Wang ,  Bradley Gene Calder ,  Yikang Xu

IPC: G06F16/00 ,  G06F16/215 ,  G06F3/06 ,  G06F12/02 ,  H04L12/26 ,  H04L29/08

Abstract: In various embodiments, methods and systems for implementing garbage collection in distributed storage systems are provided. The distributed storage system operates based on independent management of metadata of extent and stream data storage resources. A hybrid garbage collection system based on reference counting garbage collection operations and mark-and-sweep garbage collection operations is implemented. An extent lifetime table that tracks reference weights and mark sequences for extents is initialized and updated based on indications from extent managers and stream managers, respectively. Upon determining that an extent is to be handed-off from weighted reference counting garbage collection operations to mark-and-sweep garbage collection operations, a reference weight field for the extent is voided and a mark sequence field of the extent is updated. The mark sequence field is updated with a latest global sequence number. The mark-and-sweep garbage collection operations are utilized to reclaim the extent when the extent is no longer referenced.

公开(公告)号：US10754549B2

公开(公告)日：2020-08-25

申请号：US15993355

申请日：2018-05-30

Applicant: Microsoft Technology Licensing, LLC

Inventor： Bryan S. Matthew ,  Aaron W. Ogus ,  Vadim Makhervaks ,  Laura M. Caulfield ,  Rajsekhar Das ,  Scott Chao-Chueh Lee ,  Omar Carey ,  Madhav Pandya ,  Ioan Oltean ,  Garret Buban ,  Lee Prewitt

IPC: G06F3/06 ,  H03M13/15 ,  H03M13/29 ,  H03M13/00 ,  H03M13/45

Abstract: An append-only streams capability may be implemented that allows the host (e.g., the file system) to determine an optimal stream size based on the data to be stored in that stream. The storage device may expose to the host one or more characteristics of the available streams on the device, including but not limited to the maximum number of inactive and active streams on the device, the erase block size, the maximum number of erase blocks that can be written in parallel, and an optimal write size of the data. Using this information, the host may determine which particular stream offered by the device is best suited for the data to be stored.

公开(公告)号：US12073079B2

公开(公告)日：2024-08-27

申请号：US18044976

申请日：2021-09-28

Applicant: Microsoft Technology Licensing, LLC

Inventor： Scott Chao-Chueh Lee ,  Vadim Makhervaks ,  Madhav Himanshubhai Pandya ,  Ioan Oltean ,  Laura Marie Caulfield ,  Lee Edward Prewitt

IPC: G06F3/06 ,  G06F12/02

CPC classification number: G06F3/0604 ,  G06F3/061 ,  G06F3/0631 ,  G06F3/0644 ,  G06F3/0679 ,  G06F12/0246

Abstract: Zone hints for use with a zoned namespace (ZNS) storage device. Zone hints include one or more of a first hint indicating that a zone is part of a group of a plurality of zones, a second hint indicating that the zone is to be fast-filled, or a third hint indicating that the zone is associated with a background operation. The first hint is structured to instruct the ZNS storage device to allocate to the zone a first storage resources that are physically adjacent to second storage resources reserved for others of the plurality of zones. The second hint is structured to instruct the ZNS storage device to bypass a staging area when writing to the zone. The third hint is structured to instruct the ZNS storage device to deprioritizing at least one operation writing to the zone, or to bypass the staging area when writing to the zone.

公开(公告)号：US11314459B2

公开(公告)日：2022-04-26

申请号：US16811972

申请日：2020-03-06

Applicant: Microsoft Technology Licensing, LLC

Inventor： Cheng Huang ,  Andrew James Edwards ,  Shane K. Mainali ,  Aaron William Ogus ,  Ioan Oltean ,  Huseyin Simitci ,  Ju Wang ,  Bradley Gene Calder ,  Yikang Xu

IPC: G06F3/00 ,  G06F9/00 ,  G06F3/06 ,  G06F9/445 ,  G06F9/50 ,  G06F9/4401 ,  G06F11/14 ,  G06F12/02

Abstract: In various embodiments, methods and systems for implementing a distributed metadata management system in distributed storage systems are provided. A distributed storage system operates based on data storage resources (e.g., extents and streams). The distributed metadata management system is implemented for extent and stream metadata to facilitate the scalability of metadata processing. The distributed storage system implements extent managers and stream managers that independently manage extent and stream metadata, respectively. The extent managers are associated with an extent table that stores extent metadata. The stream managers are associated with streams that store associations with extents. The distributed metadata management system can also utilize a bootstrap layer that leverages components of a legacy distributed storage system to facilitate distributed management of extent and stream metadata. The bootstrap layer is used to store the extent table as a system table and to persist the state of the stream manager as system streams.

公开(公告)号：US10558565B2

公开(公告)日：2020-02-11

申请号：US15990969

申请日：2018-05-29

Applicant: Microsoft Technology Licensing, LLC

Inventor： Ioan Oltean ,  Rushi Srinivas Surla ,  Jegan Devaraju ,  Maneesh Sah ,  Julia Johnstone

IPC: G06F3/00 ,  G06F12/02 ,  G06F3/06 ,  G06F11/10

Abstract: Provided is a system and method for converting active data identified by a garbage collection operation into erasure coded fragments. In one example, the method may include identifying data blocks in use and interspersed among garbage data blocks not in use in cloud storage based on a garbage collection operation, extracting object data from the identified data blocks in use into a data container while leaving object data of the garbage data blocks not in use, and fragmenting a predetermined amount of extracted object data stored within the data container, the fragmenting comprising converting the predetermined amount of object data into a plurality of fragments including data fragments storing portions of the data and parity fragments for reconstructing the data, and writing the plurality of fragments in a distributed manner among a plurality of storage nodes.