LEARNED COMPUTER CONTROL USING POINTING DEVICE AND KEYBOARD ACTIONS

    公开(公告)号:US20250093970A1

    公开(公告)日:2025-03-20

    申请号:US18967935

    申请日:2024-12-04

    Abstract: A computer-implemented method for controlling a particular computer to execute a task is described. The method includes receiving a control input comprising a visual input, the visual input including one or more screen frames of a computer display that represent at least a current state of the particular computer; processing the control input using a neural network to generate one or more control outputs that are used to control the particular computer to execute the task, in which the one or more control outputs include an action type output that specifies at least one of a pointing device action or a keyboard action to be performed to control the particular computer; determining one or more actions from the one or more control outputs; and executing the one or more actions to control the particular computer.

    MEMORY AUGMENTED GENERATIVE TEMPORAL MODELS

    公开(公告)号:US20210089968A1

    公开(公告)日:2021-03-25

    申请号:US17113669

    申请日:2020-12-07

    Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating sequences of predicted observations, for example images. In one aspect, a system comprises a controller recurrent neural network, and a decoder neural network to process a set of latent variables to generate an observation. An external memory and a memory interface subsystem is configured to, for each of a plurality of time steps, receive an updated hidden state from the controller, generate a memory context vector by reading data from the external memory using the updated hidden state, determine a set of latent variables from the memory context vector, generate a predicted observation by providing the set of latent variables to the decoder neural network, write data to the external memory using the latent variables, the updated hidden state, or both, and generate a controller input for a subsequent time step from the latent variables.

    Augmenting neural networks with external memory

    公开(公告)号:US10885426B2

    公开(公告)日:2021-01-05

    申请号:US15396289

    申请日:2016-12-30

    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for augmenting neural networks with an external memory. One of the systems includes a controller neural network that includes a Least Recently Used Access (LRUA) subsystem configured to: maintain a respective usage weight for each of a plurality of locations in the external memory, and for each of the plurality of time steps: generate a respective reading weight for each location using a read key, read data from the locations in accordance with the reading weights, generate a respective writing weight for each of the locations from a respective reading weight from a preceding time step and the respective usage weight for the location, write a write vector to the locations in accordance with the writing weights, and update the respective usage weight from the respective reading weight and the respective writing weight.

    Learned computer control using pointing device and keyboard actions

    公开(公告)号:US12189870B2

    公开(公告)日:2025-01-07

    申请号:US18103309

    申请日:2023-01-30

    Abstract: A computer-implemented method for controlling a particular computer to execute a task is described. The method includes receiving a control input comprising a visual input, the visual input including one or more screen frames of a computer display that represent at least a current state of the particular computer; processing the control input using a neural network to generate one or more control outputs that are used to control the particular computer to execute the task, in which the one or more control outputs include an action type output that specifies at least one of a pointing device action or a keyboard action to be performed to control the particular computer; determining one or more actions from the one or more control outputs; and executing the one or more actions to control the particular computer.

    LEARNED COMPUTER CONTROL USING POINTING DEVICE AND KEYBOARD ACTIONS

    公开(公告)号:US20230244325A1

    公开(公告)日:2023-08-03

    申请号:US18103309

    申请日:2023-01-30

    CPC classification number: G06F3/033 G06F3/023 G06F40/284

    Abstract: A computer-implemented method for controlling a particular computer to execute a task is described. The method includes receiving a control input comprising a visual input, the visual input including one or more screen frames of a computer display that represent at least a current state of the particular computer; processing the control input using a neural network to generate one or more control outputs that are used to control the particular computer to execute the task, in which the one or more control outputs include an action type output that specifies at least one of a pointing device action or a keyboard action to be performed to control the particular computer; determining one or more actions from the one or more control outputs; and executing the one or more actions to control the particular computer.

Patent Agency Ranking