Data pipeline creation system and method

    公开(公告)号:US10983988B2

    公开(公告)日:2021-04-20

    申请号:US16362113

    申请日:2019-03-22

    Abstract: A computer-implemented method comprises receiving one or more data transformation commands through a console in a console session, the one or more data transformation commands relating to one or more initial datasets; executing the one or more data transformation commands using the one or more initial datasets to modify at least one of the one or more initial datasets to generate a modified dataset; generating a set of environment flags for the command to indicate that the one or more initial datasets has been accessed and the at least one dataset that has been modified; and updating a set of line dependencies based on the generated set of environmental flags and previously generated sets of environmental flags for one or more previously executed commands.

Patent Agency Ranking