-
公开(公告)号:US20240143469A1
公开(公告)日:2024-05-02
申请号:US18391229
申请日:2023-12-20
Applicant: Google LLC
Inventor: Pavan Edara , Reuven Lax , Ji Yang , Gurpreet Singh Nanda
CPC classification number: G06F11/3034 , G06F9/30047 , G06F9/467 , G06F11/0757 , G06F11/0772 , G06F11/1402 , G06F12/0246 , G06F12/0253 , G06F2201/84
Abstract: A method for processing data exactly once using transactional stream writes includes receiving, from a client, a batch of data blocks for storage on memory hardware in communication with the data processing hardware. The batch of data blocks is associated with a corresponding sequence number and represents a number of rows of a table stored on the memory hardware. The method also includes partitioning the batch of data blocks into a plurality of sub-batches of data blocks. For each sub-batch of data blocks, the method further includes assigning the sub-batch of data blocks to a buffered stream; writing, using the assigned buffered stream, the sub-batch of data blocks to the memory hardware; updating a storage log with an intent to commit the sub-batch of data blocks using the assigned buffered stream; and committing the sub-batch of data blocks to the memory hardware.
-
公开(公告)号:US11573876B2
公开(公告)日:2023-02-07
申请号:US17085576
申请日:2020-10-30
Applicant: Google LLC
Inventor: Pavan Edara , Reuven Lax , Yi Yang , Gurpreet Singh Nanda
Abstract: A method for processing data exactly once using transactional stream writes includes receiving, from a client, a batch of data blocks for storage on memory hardware in communication with the data processing hardware. The batch of data blocks is associated with a corresponding sequence number and represents a number of rows of a table stored on the memory hardware. The method also includes partitioning the batch of data blocks into a plurality of sub-batches of data blocks. For each sub-batch of data blocks, the method further includes assigning the sub-batch of data blocks to a buffered stream; writing, using the assigned buffered stream, the sub-batch of data blocks to the memory hardware; updating a storage log with an intent to commit the sub-batch of data blocks using the assigned buffered stream; and committing the sub-batch of data blocks to the memory hardware.
-
公开(公告)号:US12259800B2
公开(公告)日:2025-03-25
申请号:US18391229
申请日:2023-12-20
Applicant: Google LLC
Inventor: Pavan Edara , Reuven Lax , Ji Yang , Gurpreet Singh Nanda
Abstract: A method for processing data exactly once using transactional stream writes includes receiving, from a client, a batch of data blocks for storage on memory hardware in communication with the data processing hardware. The batch of data blocks is associated with a corresponding sequence number and represents a number of rows of a table stored on the memory hardware. The method also includes partitioning the batch of data blocks into a plurality of sub-batches of data blocks. For each sub-batch of data blocks, the method further includes assigning the sub-batch of data blocks to a buffered stream; writing, using the assigned buffered stream, the sub-batch of data blocks to the memory hardware; updating a storage log with an intent to commit the sub-batch of data blocks using the assigned buffered stream; and committing the sub-batch of data blocks to the memory hardware.
-
公开(公告)号:US20220138071A1
公开(公告)日:2022-05-05
申请号:US17085576
申请日:2020-10-30
Applicant: Google LLC
Inventor: Pavan Edara , Reuven Lax , Yi Yang , Gurpreet Singh Nanda
Abstract: A method for processing data exactly once using transactional stream writes includes receiving, from a client, a batch of data blocks for storage on memory hardware in communication with the data processing hardware. The batch of data blocks is associated with a corresponding sequence number and represents a number of rows of a table stored on the memory hardware. The method also includes partitioning the batch of data blocks into a plurality of sub-batches of data blocks. For each sub-batch of data blocks, the method further includes assigning the sub-batch of data blocks to a buffered stream; writing, using the assigned buffered stream, the sub-batch of data blocks to the memory hardware; updating a storage log with an intent to commit the sub-batch of data blocks using the assigned buffered stream; and committing the sub-batch of data blocks to the memory hardware.
-
公开(公告)号:US11880290B2
公开(公告)日:2024-01-23
申请号:US18165257
申请日:2023-02-06
Applicant: Google LLC
Inventor: Pavan Edara , Reuven Lax , Yi Yang , Gurpreet Singh Nanda
CPC classification number: G06F11/3034 , G06F9/30047 , G06F9/467 , G06F11/0757 , G06F11/0772 , G06F11/1402 , G06F12/0246 , G06F12/0253 , G06F2201/84
Abstract: A method for processing data exactly once using transactional stream writes includes receiving, from a client, a batch of data blocks for storage on memory hardware in communication with the data processing hardware. The batch of data blocks is associated with a corresponding sequence number and represents a number of rows of a table stored on the memory hardware. The method also includes partitioning the batch of data blocks into a plurality of sub-batches of data blocks. For each sub-batch of data blocks, the method further includes assigning the sub-batch of data blocks to a buffered stream; writing, using the assigned buffered stream, the sub-batch of data blocks to the memory hardware; updating a storage log with an intent to commit the sub-batch of data blocks using the assigned buffered stream; and committing the sub-batch of data blocks to the memory hardware.
-
公开(公告)号:US20230185688A1
公开(公告)日:2023-06-15
申请号:US18165257
申请日:2023-02-06
Applicant: Google LLC
Inventor: Pavan Edara , Reuven Lax , Yi Yang , Gurpreet Singh Nanda
CPC classification number: G06F11/3034 , G06F9/30047 , G06F9/467 , G06F11/0757 , G06F11/0772 , G06F12/0246 , G06F12/0253 , G06F11/1402 , G06F2201/84
Abstract: A method for processing data exactly once using transactional stream writes includes receiving, from a client, a batch of data blocks for storage on memory hardware in communication with the data processing hardware. The batch of data blocks is associated with a corresponding sequence number and represents a number of rows of a table stored on the memory hardware. The method also includes partitioning the batch of data blocks into a plurality of sub-batches of data blocks. For each sub-batch of data blocks, the method further includes assigning the sub-batch of data blocks to a buffered stream; writing, using the assigned buffered stream, the sub-batch of data blocks to the memory hardware; updating a storage log with an intent to commit the sub-batch of data blocks using the assigned buffered stream; and committing the sub-batch of data blocks to the memory hardware.
-
-
-
-
-