Automatically executing tasks and configuring access control lists in a data transformation system

    公开(公告)号:US11468083B2

    公开(公告)日:2022-10-11

    申请号:US16915693

    申请日:2020-06-29

    Abstract: A computer-implemented system or process is programmed or configured to use a configuration file to specify one or more tasks to apply to raw ingested data. A task may be a sequence of instructions programmed or configured to format raw ingested data into a dataset in a CSV format. Examples of tasks may include: a parser to parse Cobol data into a CSV, a parser to parse XML into a CSV, a parser to parse text using fixed-width fields to a CSV, a parser to parse files in a zip archive into a CSV, a regular expression search/replace function, or formatting logic to remove lines or blank lines from raw ingested data. In one embodiment, the configuration file may specify a schema definition for a task to use for generating a dataset. In one embodiment, the configuration file may also include one or more access control list (ACL) definitions for the generated dataset. In one embodiment, the building of datasets using the configuration file is automated, for example, on a nightly basis.

    SYSTEMS AND METHOD FOR AUTHENTICATING USERS OF A DATA PROCESSING PLATFORM FROM MULTIPLE IDENTITY PROVIDERS

    公开(公告)号:US20210112065A1

    公开(公告)日:2021-04-15

    申请号:US16662466

    申请日:2019-10-24

    Abstract: A system and method for authenticating users of a data processing platform stores a mapping of a unique user platform identifier to multiple user identity provider identifiers associated with multiple realms for a same user. In some examples, the method includes receiving a request from a client device to establish an access session to perform one or more actions on data of the data processing platform and receiving, from at least one of the first external identity provider of the first realm or the second external identity provider of the second realm, a user identity provider identifier associated with the request. In certain examples, the method includes granting permission to perform the one or more actions on the data of the data processing platform based at least in part on the received user identity provider identifier.

    Module assignment management
    13.
    发明授权

    公开(公告)号:US10409641B1

    公开(公告)日:2019-09-10

    申请号:US16250829

    申请日:2019-01-17

    Abstract: Systems, methods, and non-transitory computer readable media are provided for managing assignment of modules. A job from a client may be received. The job may be inserted into a job queue. The job within the job queue may be compared with a set of cooldown modules to determine whether a compatible cooldown module is available. The job may be assigned to the compatible cooldown module responsive to the compatible cooldown module being available.

    AUTOMATICALLY EXECUTING TASKS AND CONFIGURING ACCESS CONTROL LISTS IN A DATA TRANSFORMATION SYSTEM

    公开(公告)号:US20180181632A1

    公开(公告)日:2018-06-28

    申请号:US15845879

    申请日:2017-12-18

    Abstract: A computer-implemented system or process is programmed or configured to use a configuration file to specify one or more tasks to apply to raw ingested data. A task may be a sequence of instructions programmed or configured to format raw ingested data into a dataset in a CSV format. Examples of tasks may include: a parser to parse Cobol data into a CSV, a parser to parse XML into a CSV, a parser to parse text using fixed-width fields to a CSV, a parser to parse files in a zip archive into a CSV, a regular expression search/replace function, or formatting logic to remove lines or blank lines from raw ingested data. In one embodiment, the configuration file may specify a schema definition for a task to use for generating a dataset. In one embodiment, the configuration file may also include one or more access control list (ACL) definitions for the generated dataset. In one embodiment, the building of datasets using the configuration file is automated, for example, on a nightly basis.

    Project-based permission system
    15.
    发明授权

    公开(公告)号:US12299106B2

    公开(公告)日:2025-05-13

    申请号:US18109134

    申请日:2023-02-13

    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media for data security protection are provided. One of the methods includes: receiving a job associated with a project, wherein the project is associated with one or more data sources; identifying a plurality of inputs and a plurality of outputs associated with the job; determining a plurality of required permissions associated with the job, wherein each of the required permissions comprises an operation on a required data source, the operation corresponding to at least one of the inputs or the outputs; verifying that the one or more data sources associated with the project comprise the required data source associated with each of the required permissions; and generating a token associated with the job, the token encoding the required permissions associated with the job, wherein the token is required for execution of the job.

    SECURE HIGH SCALE CRYPTOGRAPHIC COMPUTATION THROUGH DELEGATED KEY ACCESS

    公开(公告)号:US20230418953A1

    公开(公告)日:2023-12-28

    申请号:US17846648

    申请日:2022-06-22

    CPC classification number: G06F21/602 H04L9/088

    Abstract: An apparatus, computer-implemented method and computer program are disclosed for performing a cryptographic operation in a high-trust (HT) environment. The HT environment including a compute service and key storage service. The compute service receives from a user device, a user request for performing a cryptographic operation on at least a portion of a large-scale dataset. The user request including a user token associated with a user of the user device. The compute service sends to the key storage service, a cryptographic key access request corresponding to the received user request. The cryptographic key access request including data representative of the user token and/or a compute service token. The key storage service determines from the user token and/or compute service token whether the user has permission to have the cryptographic operation performed and/or whether to grant the compute service access to data representative of the cryptographic key in relation to the requested cryptographic operation when user has permission. In response to the key storage service granting access to the compute service, the key storage service sends to the compute service the requested cryptographic key/algorithm associated with the cryptographic operation of the user request. The compute service performs the cryptographic operation on the portion of the large-scale dataset based on the received cryptographic key/algorithm.

    PROJECTIONS FOR BIG DATABASE SYSTEMS
    19.
    发明公开

    公开(公告)号:US20230281191A1

    公开(公告)日:2023-09-07

    申请号:US18178294

    申请日:2023-03-03

    CPC classification number: G06F16/2379 G06F16/273 G06F16/2455 G06F16/248

    Abstract: A database system comprised of a decoupled compute layer and storage layer is implemented to store, build, and maintain a canonical dataset, a temporary buffer, and projection datasets. The canonical dataset is a set of batch updated data. The data is appended in chunks to the canonical dataset such that the canonical dataset becomes a historical dataset over time. The buffer is a write ahead log that contains the most recent chunks of data and provides atomicity and durability for the database system. The projection datasets are indexes of the canonical dataset and/or the buffer that may have single or multiple column sort-orders and/or particular data formats. The writes to the canonical dataset, projection datasets, and buffer may be asynchronous and therefore the database system is advantageously less resource constrained.

Patent Agency Ranking