Storing B-tree pages in capacity tier for erasure-coded storage in distributed data systems

    公开(公告)号:US11144394B1

    公开(公告)日:2021-10-12

    申请号:US16894646

    申请日:2020-06-05

    Applicant: VMware, Inc.

    Abstract: Systems, for managing metadata data blocks, include a fast performance tier and a large capacity tier. The capacity tier stores the data blocks in erasure-encoded stripes. The performance tier stores map data indicating correspondences between logical addresses, associated with a first layer of the system, and physical addresses associated with a second layer. The map data is structured as a B-tree with leaf nodes and index nodes. A method includes determining a volume associated with the leaves. In response to the volume being greater than a threshold, migrating a portion of the leaves to the stripes of the capacity tier. A portion of the indexes that include pointers to the migrated portion of the leaves are updated to include updated pointers to physical addresses of the stripes that store the migrated leaves. A portion of the performance tier that stores the migrated leaves is re-allocated to store additional map data.

    Data channel between a client and a restartable service

    公开(公告)号:US11088896B2

    公开(公告)日:2021-08-10

    申请号:US15652048

    申请日:2017-07-17

    Applicant: VMware, Inc.

    Abstract: A data communication channel between a client and a service is preserved through a failure of the server by maintaining a request log and an inflight request queue in a protected memory region that preserves the contents of the request log and the inflight request queue even when the service encounters a failure. The method of restarting the data communication channel includes, upon the service being restarted following the failure of the service, determining whether the request log contains requests and, if so, copying the requests from the request log into the in-flight request queue and then removing the copied requests from the request log. The requests in the in-flight request queue, which include any that were in the in-flight request queue at the time of the failure of the service and any that were copied from the request log, are then processed.

    Enhanced data encryption in distributed datastores using a cluster-wide fixed random tweak

    公开(公告)号:US11061594B1

    公开(公告)日:2021-07-13

    申请号:US16827674

    申请日:2020-03-23

    Applicant: VMware, Inc.

    Abstract: A method for encrypting data in one or more data blocks is provided. The method generates a fixed random tweak. The method receives first and second data blocks to write on at least one physical disk of a set of physical disks associated with a set of host machines. The method applies a fixed random tweak to data indicative of the first data block and data indicative of the second data block to generate, respectively, first and second encrypted data blocks. The method writes first and second entries to a data log in a cache, the first entry comprising a first header and the first encrypted data block and the second entry comprising a second header and the second encrypted data block. The method then writes the first and second encrypted data blocks to the at least one physical disk.

    Scalable space management using bitmap summary tree

    公开(公告)号:US10528459B2

    公开(公告)日:2020-01-07

    申请号:US14713698

    申请日:2015-05-15

    Applicant: VMware, Inc.

    Abstract: Systems and methods for allocating space in persistent storage are provided. A modified bitmap and a tree of bitmap summary pages are used to manage the free space of a large scale storage system. The bitmap is separated into fixed size pages and has bitmap summary entries to summarize the information in the bitmap. Bitmap summary pages can be further summarized into secondary summary pages. The tree data structure can continue to N levels until a topmost level has one bitmap summary page.

    SYSTEM AND METHOD FOR MANAGING STORAGE TRANSACTION REQUESTS

    公开(公告)号:US20190108104A1

    公开(公告)日:2019-04-11

    申请号:US15727211

    申请日:2017-10-06

    Applicant: VMware, Inc.

    Abstract: Data storage system and method for managing transaction requests to the data storage system utilizes a write ahead log to write transaction requests received at the data storage system during a current checkpoint generation. After the transaction requests in the write ahead log are applied to a copy-on-write (COW) storage data structure stored in a storage system, one of first and second allocation bitmaps is updated to reflect changes in the COW storage data structure with respect to allocation of storage space in the storage system, and one of first and second super blocks is updated with references to central nodes of the COW storage data structure. After the allocation bitmap and the super block have been updated, an end indicator for the current checkpoint generation is written in the write ahead log to indicate that processing of the transaction requests for the current checkpoint generation has been completed.

    Abortable transactions using versioned tuple cache

    公开(公告)号:US10176209B2

    公开(公告)日:2019-01-08

    申请号:US14752175

    申请日:2015-06-26

    Applicant: VMware, Inc.

    Abstract: A transaction manager for handling operations on data in a storage system provides a system for executing transactions that uses a versioned tuple cache to achieve fast, abortable transactions using a redo-only log. The transaction manager updates an in-memory key-value store and also attaches a transaction identifier to the tuple as a minor key. Opportunistic locking can be accomplished due to the low cost of aborting transactions.

    Shared Data Cache for Kernel Bypass Applications

    公开(公告)号:US20180357176A1

    公开(公告)日:2018-12-13

    申请号:US15622014

    申请日:2017-06-13

    Applicant: VMware, Inc.

    Inventor: Wenguang Wang

    CPC classification number: G06F12/0888 G06F12/084 G06F12/1063

    Abstract: Techniques for implementing a shared data cache for kernel bypass applications are provided. In one set of embodiments, a shared data caching (SDC) service associated with an instance of a kernel bypass application can create a named shared memory region in user space, where the kernel bypass application is configured to use a user-level Input/Output (I/O) stack for accessing a physical I/O device. The SDC service can further map the named shared memory region into a virtual memory address space of the instance. Then, at a time the instance issues an I/O read request to the physical I/O device, the SDC service can process the I/O read request by accessing the named shared memory region as a data cache.

Patent Agency Ranking