公开(公告)号：US11526680B2

公开(公告)日：2022-12-13

申请号：US16790917

申请日：2020-02-14

Applicant: Google LLC

Inventor： Sujith Ravi ,  Zornitsa Kozareva ,  Chinnadhurai Sankar

IPC: G06F40/56 ,  G06N20/00 ,  H04L67/04 ,  G06K9/62 ,  G06N3/04

Abstract: Systems and methods are provided to pre-train projection networks for use as transferable natural language representation generators. In particular, example pre-training schemes described herein enable learning of transferable deep neural projection representations over randomized locality sensitive hashing (LSH) projections, thereby surmounting the need to store any embedding matrices because the projections can be dynamically computed at inference time.

公开(公告)号：US12020706B2

公开(公告)日：2024-06-25

申请号：US17899162

申请日：2022-08-30

Applicant: GOOGLE LLC

Inventor： Arvind Neelakantan ,  Daniel Duckworth ,  Ben Goodrich ,  Vishaal Prasad ,  Chinnadhurai Sankar ,  Semih Yavuz

IPC: G10L15/22 ,  G06N5/04

CPC classification number: G10L15/22 ,  G06N5/04 ,  G10L2015/225

Abstract: Training and/or utilizing a single neural network model to generate, at each of a plurality of assistant turns of a dialog session between a user and an automated assistant, a corresponding automated assistant natural language response and/or a corresponding automated assistant action. For example, at a given assistant turn of a dialog session, both a corresponding natural language response and a corresponding action can be generated jointly and based directly on output generated using the single neural network model. The corresponding response and/or corresponding action can be generated based on processing, using the neural network model, dialog history and a plurality of discrete resources. For example, the neural network model can be used to generate a response and/or action on a token-by-token basis.

公开(公告)号：US20200265196A1

公开(公告)日：2020-08-20

申请号：US16790917

申请日：2020-02-14

Applicant: Google LLC

Inventor： Sujith Ravi ,  Zornitsa Kozareva ,  Chinnadhurai Sankar

IPC: G06F40/56 ,  G06N20/00 ,  G06N3/04 ,  G06K9/62 ,  H04L29/08

Abstract: Systems and methods are provided to pre-train projection networks for use as transferable natural language representation generators. In particular, example pre-training schemes described herein enable learning of transferable deep neural projection representations over randomized locality sensitive hashing (LSH) projections, thereby surmounting the need to store any embedding matrices because the projections can be dynamically computed at inference time.

公开(公告)号：US20240347061A1

公开(公告)日：2024-10-17

申请号：US18751911

申请日：2024-06-24

Applicant: GOOGLE LLC

Inventor： Arvind Neelakantan ,  Daniel Duckworth ,  Ben Goodrich ,  Vishaal Prasad ,  Chinnadhurai Sankar ,  Semih Yavuz

IPC: G10L15/22 ,  G06N5/04

CPC classification number: G10L15/22 ,  G06N5/04 ,  G10L2015/225

Abstract: Training and/or utilizing a single neural network model to generate, at each of a plurality of assistant turns of a dialog session between a user and an automated assistant, a corresponding automated assistant natural language response and/or a corresponding automated assistant action. For example, at a given assistant turn of a dialog session, both a corresponding natural language response and a corresponding action can be generated jointly and based directly on output generated using the single neural network model. The corresponding response and/or corresponding action can be generated based on processing, using the neural network model, dialog history and a plurality of discrete resources. For example, the neural network model can be used to generate a response and/or action on a token-by-token basis.

公开(公告)号：US20220415324A1

公开(公告)日：2022-12-29

申请号：US17899162

申请日：2022-08-30

Applicant: GOOGLE LLC

Inventor： Arvind Neelakantan ,  Daniel Duckworth ,  Ben Goodrich ,  Vishaal Prasad ,  Chinnadhurai Sankar ,  Semih Yavuz

IPC: G10L15/22 ,  G06N5/04

Abstract: Training and/or utilizing a single neural network model to generate, at each of a plurality of assistant turns of a dialog session between a user and an automated assistant, a corresponding automated assistant natural language response and/or a corresponding automated assistant action. For example, at a given assistant turn of a dialog session, both a corresponding natural language response and a corresponding action can be generated jointly and based directly on output generated using the single neural network model. The corresponding response and/or corresponding action can be generated based on processing, using the neural network model, dialog history and a plurality of discrete resources. For example, the neural network model can be used to generate a response and/or action on a token-by-token basis.

公开(公告)号：US11475890B2

公开(公告)日：2022-10-18

申请号：US16910435

申请日：2020-06-24

Applicant: Google LLC

Inventor： Arvind Neelakantan ,  Daniel Duckworth ,  Ben Goodrich ,  Vishaal Prasad ,  Chinnadhurai Sankar ,  Semih Yavuz

IPC: G10L15/22 ,  G06N5/04

Abstract: Training and/or utilizing a single neural network model to generate, at each of a plurality of assistant turns of a dialog session between a user and an automated assistant, a corresponding automated assistant natural language response and/or a corresponding automated assistant action. For example, at a given assistant turn of a dialog session, both a corresponding natural language response and a corresponding action can be generated jointly and based directly on output generated using the single neural network model. The corresponding response and/or corresponding action can be generated based on processing, using the neural network model, dialog history and a plurality of discrete resources. For example, the neural network model can be used to generate a response and/or action on a token-by-token basis.