公开(公告)号：US20210334320A1

公开(公告)日：2021-10-28

申请号：US17280027

申请日：2019-09-27

Applicant: Google LLC

Inventor： Aleksandra Faust ,  Dilek Hakkani-Tur ,  Izzeddin Gur ,  Ulrich Rueckert

IPC: G06F16/954 ,  G06N3/04 ,  G06F16/953

Abstract: The present disclosure is generally directed to methods, apparatus, and computer-readable media (transitory and non-transitory) for learning to automatically navigate interactive web documents and/or websites. More particularly, various approaches are presented for training various deep Q network (DQN) agents to perform various tasks associated with reinforcement learning, including hierarchical reinforcement learning, in challenging web navigation environments with sparse rewards and large state and action spaces. These agents include a web navigation agent that can use learned value function(s) to automatically navigate through interactive web documents, as well as a training agent, referred to herein as a “meta-trainer,” that can be trained to generate synthetic training examples. Some approaches described herein may be implemented when expert demonstrations are available. Other approaches described herein may be implemented when expert demonstrations are not available. In either case, dense, potential-based rewards may be used to augment the training.

公开(公告)号：US12118052B2

公开(公告)日：2024-10-15

申请号：US18234766

申请日：2023-08-16

Applicant: GOOGLE LLC

Inventor： Aleksandra Faust ,  Dilek Hakkani-Tur ,  Izzeddin Gur ,  Ulrich Rueckert

IPC: G06F16/954 ,  G06F16/953 ,  G06N3/04

CPC classification number: G06F16/954 ,  G06F16/953 ,  G06N3/04

Abstract: The present disclosure is generally directed to methods, apparatus, and computer-readable media (transitory and non-transitory) for learning to automatically navigate interactive web documents and/or websites. More particularly, various approaches are presented for training various deep Q network (DQN) agents to perform various tasks associated with reinforcement learning, including hierarchical reinforcement learning, in challenging web navigation environments with sparse rewards and large state and action spaces. These agents include a web navigation agent that can use learned value function(s) to automatically navigate through interactive web documents, as well as a training agent, referred to herein as a “meta-trainer,” that can be trained to generate synthetic training examples. Some approaches described herein may be implemented when expert demonstrations are available. Other approaches described herein may be implemented when expert demonstrations are not available. In either case, dense, potential-based rewards may be used to augment the training.

公开(公告)号：US20230394102A1

公开(公告)日：2023-12-07

申请号：US18234766

申请日：2023-08-16

Applicant: GOOGLE LLC

Inventor： Aleksandra Faust ,  Dilek Hakkani-Tur ,  Izzeddin Gur ,  Ulrich Rueckert

IPC: G06F16/954 ,  G06F16/953 ,  G06N3/04

CPC classification number: G06F16/954 ,  G06F16/953 ,  G06N3/04

Abstract: The present disclosure is generally directed to methods, apparatus, and computer-readable media (transitory and non-transitory) for learning to automatically navigate interactive web documents and/or websites. More particularly, various approaches are presented for training various deep Q network (DQN) agents to perform various tasks associated with reinforcement learning, including hierarchical reinforcement learning, in challenging web navigation environments with sparse rewards and large state and action spaces. These agents include a web navigation agent that can use learned value function(s) to automatically navigate through interactive web documents, as well as a training agent, referred to herein as a “meta-trainer,” that can be trained to generate synthetic training examples. Some approaches described herein may be implemented when expert demonstrations are available. Other approaches described herein may be implemented when expert demonstrations are not available. In either case, dense, potential-based rewards may be used to augment the training.

公开(公告)号：US20250077603A1

公开(公告)日：2025-03-06

申请号：US18952242

申请日：2024-11-19

Applicant: GOOGLE LLC

Inventor： Aleksandra Faust ,  Dilek Hakkani-Tur ,  Izzeddin Gur ,  Ulrich Rueckert

IPC: G06F16/954 ,  G06F16/953 ,  G06N3/04

Abstract: The present disclosure is generally directed to methods, apparatus, and computer-readable media (transitory and non-transitory) for learning to automatically navigate interactive web documents and/or websites. More particularly, various approaches are presented for training various deep Q network (DQN) agents to perform various tasks associated with reinforcement learning, including hierarchical reinforcement learning, in challenging web navigation environments with sparse rewards and large state and action spaces. These agents include a web navigation agent that can use learned value function(s) to automatically navigate through interactive web documents, as well as a training agent, referred to herein as a “meta-trainer,” that can be trained to generate synthetic training examples. Some approaches described herein may be implemented when expert demonstrations are available. Other approaches described herein may be implemented when expert demonstrations are not available. In either case, dense, potential-based rewards may be used to augment the training.

公开(公告)号：US12153642B2

公开(公告)日：2024-11-26

申请号：US18234766

申请日：2023-08-16

Applicant: GOOGLE LLC

Inventor： Aleksandra Faust ,  Dilek Hakkani-Tur ,  Izzeddin Gur ,  Ulrich Rueckert

IPC: G06F16/954 ,  G06F16/953 ,  G06N3/04

Abstract: The present disclosure is generally directed to methods, apparatus, and computer-readable media (transitory and non-transitory) for learning to automatically navigate interactive web documents and/or websites. More particularly, various approaches are presented for training various deep Q network (DQN) agents to perform various tasks associated with reinforcement learning, including hierarchical reinforcement learning, in challenging web navigation environments with sparse rewards and large state and action spaces. These agents include a web navigation agent that can use learned value function(s) to automatically navigate through interactive web documents, as well as a training agent, referred to herein as a “meta-trainer,” that can be trained to generate synthetic training examples. Some approaches described herein may be implemented when expert demonstrations are available. Other approaches described herein may be implemented when expert demonstrations are not available. In either case, dense, potential-based rewards may be used to augment the training.

公开(公告)号：US11734375B2

公开(公告)日：2023-08-22

申请号：US17280027

申请日：2019-09-27

Applicant: Google LLC

Inventor： Aleksandra Faust ,  Dilek Hakkani-Tur ,  Izzeddin Gur ,  Ulrich Rueckert

IPC: G06F16/954 ,  G06F16/953 ,  G06N3/04

CPC classification number: G06F16/954 ,  G06F16/953 ,  G06N3/04

Abstract: The present disclosure is generally directed to methods, apparatus, and computer-readable media (transitory and non-transitory) for learning to automatically navigate interactive web documents and/or websites. More particularly, various approaches are presented for training various deep Q network (DQN) agents to perform various tasks associated with reinforcement learning, including hierarchical reinforcement learning, in challenging web navigation environments with sparse rewards and large state and action spaces. These agents include a web navigation agent that can use learned value function(s) to automatically navigate through interactive web documents, as well as a training agent, referred to herein as a “meta-trainer,” that can be trained to generate synthetic training examples. Some approaches described herein may be implemented when expert demonstrations are available. Other approaches described herein may be implemented when expert demonstrations are not available. In either case, dense, potential-based rewards may be used to augment the training.