公开(公告)号：US20230409677A1

公开(公告)日：2023-12-21

申请号：US17841022

申请日：2022-06-15

Applicant: X Development LLC

Inventor： David Andre ,  Yu-Ann Madan

IPC: G06K9/62 ,  G06N7/00

CPC classification number: G06K9/6277 ,  G06K9/6252 ,  G06K9/6215 ,  G06N7/005

Abstract: Disclosed implementations relate to automatically generating and providing guidance for navigating HCIs to carry out semantically equivalent/similar computing tasks across different computer applications. In various implementations, a domain of a first computer application that is operable using a first HCI may be used to select a domain model that translates between an action space of the first computer application and another space. Based on the selected domain model, a domain-agnostic action embedding—representing actions performed previously using a second HCI of a second computer application to perform a semantic task—may be processed to generate probability distribution(s) over actions in the action space of the first computer application. Based on the probability distribution(s), actions may be identified that are performable using the first computer application—these actions may be used to generate guidance for navigating the first HCI to perform the semantic task.

公开(公告)号：US20230359789A1

公开(公告)日：2023-11-09

申请号：US18142472

申请日：2023-05-02

Applicant: X Development LLC

Inventor： David Andre ,  Rishabh Singh ,  Rebecca Radkoff ,  Yu-Ann Madan ,  Nisarg Vyas ,  Jayendra Parmar ,  Falak Shah ,  Shaili Trivedi

IPC: G06F30/27 ,  G10L15/183

CPC classification number: G06F30/27 ,  G10L15/183

Abstract: As opposed to a rigid approach, implementations disclosed herein utilize a flexible approach in automatically determining an action set to utilize in attempting performance of a task that is requested by natural language input of a user. The approach is flexible at least in that embedding technique(s) and/or action model(s), that are utilized in generating action set(s) from which the action set to utilize is determined, are at least selectively varied. Put another way, implementations leverage a framework via which different embedding technique(s) and/or different action model(s) can at least selectively be utilized in generating different candidate action sets for given NL input of a user. Further, one of those action sets can be selected for actual use in attempting real-world performance of a given task reflected by the given NL input. The selection can be based on a suitability metric for the selected action set and/or other considerations.