-
公开(公告)号:US11853274B2
公开(公告)日:2023-12-26
申请号:US17971482
申请日:2022-10-21
Applicant: Snowflake Inc.
IPC: G06F16/00 , G06F16/215 , G06F16/248 , G06F16/2455 , G06F16/2457
CPC classification number: G06F16/215 , G06F16/248 , G06F16/24552 , G06F16/24573
Abstract: Disclosed are techniques for deduplicating files to be ingested by a database. A bloom filter may be built for each of a first set of files to be ingested into a data exchange to generate a set of bloom filters, wherein the data exchange includes a metadata storage where metadata including a list of files ingested is stored. The set of bloom filters may be stored in the metadata storage of the data exchange. In response to receiving a set of candidate files to be ingested into the data exchange, the set of bloom filters may be used to identify from within the set of candidate files, each candidate file that is duplicative of a file in the first set of files.
-
公开(公告)号:US20230315700A1
公开(公告)日:2023-10-05
申请号:US17971482
申请日:2022-10-21
Applicant: Snowflake Inc.
IPC: G06F16/215 , G06F16/248 , G06F16/2455 , G06F16/2457
CPC classification number: G06F16/215 , G06F16/248 , G06F16/24552 , G06F16/24573
Abstract: Embodiments of the present disclosure provide techniques for deduplicating files to be ingested by a database. A bloom filter may be built for each of a first set of files to be ingested into a data exchange to generate a set of bloom filters, wherein the data exchange includes a metadata storage where metadata including a list of files ingested is stored. The set of bloom filters may be stored in the metadata storage of the data exchange. In response to receiving a set of candidate files to be ingested into the data exchange, the set of bloom filters may be used to identify from within the set of candidate files, each candidate file that is duplicative of a file in the first set of files.
-
公开(公告)号:US20230252039A1
公开(公告)日:2023-08-10
申请号:US18302515
申请日:2023-04-18
Applicant: Snowflake Inc.
CPC classification number: G06F16/254 , G06F16/2282 , G06F16/258
Abstract: Techniques for configuring event tables include retrieving, by at least one hardware processor of a computing node, log data at a first process of the computing node. The log data is associated with a function executing at a second process of the computing node. The log data is revised using a table stage to generate revised log data. The table stage is configured as a dedicated storage location of the first process. The revising includes a data enrichment process based on metadata associated with execution of the function at the second process. The revised log data is ingested into an event table.
-
公开(公告)号:US11645298B1
公开(公告)日:2023-05-09
申请号:US17934857
申请日:2022-09-23
Applicant: Snowflake Inc.
CPC classification number: G06F16/254 , G06F16/2282 , G06F16/258
Abstract: Techniques for configuring managed event tables include generating at a first process of an execution node, log data associated with execution of user-defined function (UDF) code. The log data is provided from the first process to a second process of the execution node. The first process is configured as a sub-process of the second process. The log data is formatted using the second process of the execution node, to generate formatted log data. The formatting is based on a configuration of a managed event table that is external to the execution node. The formatted log data is communicated from the second process of the execution node into a managed event table maintained at a computing node that is external to the execution node.
-
公开(公告)号:US20220350880A1
公开(公告)日:2022-11-03
申请号:US17326646
申请日:2021-05-21
Applicant: Snowflake Inc.
Inventor: Thierry Cruanes , Ganeshan Ramachandran Iyer , Isaac Kunen
IPC: G06F21/54 , G06F16/2455 , G06F21/53 , G06F21/60
Abstract: The logging techniques described herein can enable using logging tools without having to use different methods for sandbox implementations and push out the log data to storage without problems. The log data is treated as sensitive data and is protected according to the defined security policies. Further, the results may be compressed and encrypted.
-
公开(公告)号:US11487870B1
公开(公告)日:2022-11-01
申请号:US17326646
申请日:2021-05-21
Applicant: Snowflake Inc.
Inventor: Thierry Cruanes , Ganeshan Ramachandran Iyer , Isaac Kunen
IPC: G06F21/54 , G06F16/2455 , G06F21/60 , G06F21/53
Abstract: The logging techniques described herein can enable using logging tools without having to use different methods for sandbox implementations and push out the log data to storage without problems. The log data is treated as sensitive data and is protected according to the defined security policies. Further, the results may be compressed and encrypted.
-
公开(公告)号:US11487778B1
公开(公告)日:2022-11-01
申请号:US17649571
申请日:2022-02-01
Applicant: Snowflake Inc.
Abstract: Provided herein are systems and methods for configuring managed event tables. A system includes at least one hardware processor coupled to a memory and configured to collect using an event table instance within a sandbox process, log data generated by a UDF during execution of the UDF code. The log data is provided from the sandbox process to an execution node process of the execution node. The log data is formatted using the execution node process, to generate formatted log data. The formatting is based on a configuration of a managed event table that is external to the execution node. The at least one hardware processor further causes ingestion of the formatted log data from the execution node process into the managed event table.
-
公开(公告)号:US20210311957A1
公开(公告)日:2021-10-07
申请号:US16842243
申请日:2020-04-07
Applicant: Snowflake Inc.
Inventor: Gjorgji Georgievski , Ganeshan Ramachandran Iyer , Dinesh Chandrakant Kulkarni , Jiaxing Liang , Subramanian Muralidhar
Abstract: Embodiments of the present disclosure may provide cross cloud auto-ingestion techniques. A deployment may monitor multiple queues across different cloud providers and may classify the queues based on their cloud. provider type. The deployment may receive notifications from those queues regarding new data ready for ingestion. The deployment may maintain a pool of credentials and may assign appropriate credentials to each queue. The deployment may route the notifications to appropriate receivers based on cloud provider types. The receivers may then auto-ingest new data in the corresponding queue.
-
-
-
-
-
-
-