公开(公告)号：US12046019B2

公开(公告)日：2024-07-23

申请号：US17157392

申请日：2021-01-25

Applicant: SAP SE

Inventor： Mithilesh Kumar Singh ,  Anubhav Sadana ,  Deepak Pandian ,  Raghavendra D ,  Satyadeep Dey ,  Philippe Long

IPC: G06V30/40 ,  G06F9/451 ,  G06F18/22 ,  G06V10/75 ,  G06V20/62 ,  G06V30/10

CPC classification number: G06V10/751 ,  G06F9/451 ,  G06F18/22 ,  G06V20/62 ,  G06V30/10

Abstract: Disclosed herein are system, method, and computer program product embodiments for surface automation in black box environments. An embodiment operates by determining scenarios of an application for automation; detecting the scenario during an execution of an application; capturing and storing one or more user interface screenshots of the scenario; identifying and storing user interface information from the user interface screenshot; implementing a sequential set of instructions comprising at least one textual element detection technique and at least one non-textual element detection technique; and executing the sequential set of instructions.

公开(公告)号：US10956295B1

公开(公告)日：2021-03-23

申请号：US16802144

申请日：2020-02-26

Applicant: SAP SE

Inventor： Philippe Long ,  Alexandre Fortin

IPC: G06F11/34 ,  G06F9/451

Abstract: Systems, methods, and computer program products are described herein for automatically recognizing user interface (UI) elements. Capturing, by a first application, of user interactions with a graphical user interface (GUI) of a second application is initiated. The GUI includes a plurality of UI elements. The first application is separate and distinct from the second application. A selection of a UI element is received. The selection is iteratively evaluated against a ranked criteria list having a plurality of criteria to identify whether the selection matches at least one criterion of the plurality of criteria. The selection is appended to the ranked criteria list based on the selection matching the at least one criterion. A second selection of another UI element is received from the second application. The first application automatically appends a criterion to the second selection based on the ranked criteria list.