公开(公告)号：US12141879B2

公开(公告)日：2024-11-12

申请号：US17500679

申请日：2021-10-13

Applicant: META PLATFORMS TECHNOLOGIES, LLC

Inventor： Xiaohu Liu ,  Baiyang Liu ,  Rajen Subba

IPC: G06Q50/00 ,  G06F3/01 ,  G06F9/451 ,  G06F9/48 ,  G06F9/54 ,  G06F16/332 ,  G06F16/9032 ,  G06F16/9536 ,  G06F18/2321 ,  G06F40/205 ,  G06F40/242 ,  G06F40/253 ,  G06F40/295 ,  G06F40/30 ,  G06F40/35 ,  G06F40/56 ,  G06N3/04 ,  G06N3/045 ,  G06N3/047 ,  G06N3/08 ,  G06N20/00 ,  G06Q10/109 ,  G06Q30/0601 ,  G06V10/20 ,  G06V10/764 ,  G06V10/82 ,  G06V20/00 ,  G06V20/20 ,  G06V20/30 ,  G06V40/16 ,  G06V40/20 ,  G10L15/06 ,  G10L15/08 ,  G10L15/16 ,  G10L15/18 ,  G10L15/22 ,  G10L15/30 ,  G10L15/32 ,  H04L51/18 ,  H04L51/212 ,  H04L51/222 ,  H04L51/224 ,  H04L51/52 ,  H04L67/306 ,  H04L67/75 ,  H04N7/14 ,  G06F3/16 ,  G06V20/40

Abstract: In one embodiment, a method includes establishing a video call between a plurality of client systems, wherein access to an assistant system is persistently maintained during the video call, receiving, from a first client system of the plurality of client systems, a request by a first user to be performed by the assistant system during the video call, wherein the request references one or more activities associated with one or more users associated with the plurality of client systems, analyzing, by a context engine of the assistant system, images of a scene of the video call to identify the one or more activities within the scene, instructing the assistant system to execute the request based on the identified one or more activities, and sending, to one or more of the plurality of client systems, a response to the request while maintaining the video call between the plurality of client systems.

公开(公告)号：US20240104668A1

公开(公告)日：2024-03-28

申请号：US18514597

申请日：2023-11-20

Applicant: Meta Platforms Technologies, LLC

Inventor： Xiaohu Liu ,  Zhiguang Wang ,  Rajen Subba

IPC: G06Q50/00 ,  G06F3/01 ,  G06F9/451 ,  G06F9/48 ,  G06F9/54 ,  G06F16/332 ,  G06F16/9032 ,  G06F16/9536 ,  G06F18/2321 ,  G06F40/205 ,  G06F40/242 ,  G06F40/253 ,  G06F40/30 ,  G06F40/35 ,  G06F40/56 ,  G06N3/045 ,  G06N3/047 ,  G06N3/08 ,  G06N20/00 ,  G06Q10/109 ,  G06V10/20 ,  G06V10/764 ,  G06V10/82 ,  G06V20/00 ,  G06V20/20 ,  G06V20/30 ,  G06V40/16 ,  G06V40/20 ,  G10L15/06 ,  G10L15/08 ,  G10L15/16 ,  G10L15/18 ,  G10L15/22 ,  G10L15/30 ,  G10L15/32 ,  H04L51/18 ,  H04L51/212 ,  H04L51/222 ,  H04L51/224 ,  H04L51/52 ,  H04L67/306 ,  H04L67/75 ,  H04N7/14

CPC classification number: G06Q50/01 ,  G06F3/011 ,  G06F3/013 ,  G06F9/453 ,  G06F9/485 ,  G06F9/4881 ,  G06F9/547 ,  G06F16/3329 ,  G06F16/90332 ,  G06F16/9536 ,  G06F18/2321 ,  G06F40/205 ,  G06F40/242 ,  G06F40/253 ,  G06F40/30 ,  G06F40/35 ,  G06F40/56 ,  G06N3/045 ,  G06N3/047 ,  G06N3/08 ,  G06N20/00 ,  G06Q10/109 ,  G06V10/255 ,  G06V10/764 ,  G06V10/82 ,  G06V20/00 ,  G06V20/20 ,  G06V20/30 ,  G06V40/16 ,  G06V40/25 ,  G10L15/063 ,  G10L15/08 ,  G10L15/16 ,  G10L15/1815 ,  G10L15/1822 ,  G10L15/22 ,  G10L15/30 ,  G10L15/32 ,  H04L51/18 ,  H04L51/212 ,  H04L51/222 ,  H04L51/224 ,  H04L51/52 ,  H04L67/306 ,  H04L67/75 ,  H04N7/147 ,  G06F3/017 ,  G06F3/167 ,  G10L2015/088 ,  G10L2015/223 ,  G10L2015/227

Abstract: In one embodiment, a method includes receiving a first user request at a client system to suspend a first task being executed by an assistant system operating on the client system, suspending the execution of the first task responsive to the first user request, receiving a second user request at the client system, determining that the second user request is a request to resume the suspended first task based on user interactions with the assistant system with respect to one or more entities associated with the first task, and presenting a prompt to resume the first task at the client system.

公开(公告)号：US20230419651A1

公开(公告)日：2023-12-28

申请号：US18449525

申请日：2023-08-14

Applicant: Meta Platforms Technologies, LLC

Inventor： Paul Anthony Crook ,  Francislav P. Penov ,  Rajen Subba ,  Xiaohu Liu

IPC: G06V10/82 ,  G06F9/451 ,  G10L15/18 ,  G10L15/183 ,  G10L15/22 ,  G06F16/338 ,  G06F16/332 ,  G06F16/33 ,  G06N20/00 ,  G06F16/9535 ,  G06Q50/00 ,  H04L67/306 ,  G06F16/176 ,  G10L15/06 ,  G10L15/16 ,  G06F3/01 ,  G06F16/9032 ,  G06F16/2457 ,  H04L51/02 ,  G06F3/16 ,  G06N3/08 ,  G10L15/26 ,  G06F16/9038 ,  G06F16/904 ,  G06F40/30 ,  G06F40/40 ,  G06F16/22 ,  G06F16/23 ,  G06F7/14 ,  H04L43/0882 ,  H04L43/0894 ,  H04L12/28 ,  H04L41/00 ,  H04L41/22 ,  H04W12/08 ,  G10L15/07 ,  G10L17/22 ,  G06N3/006 ,  G10L17/06 ,  G06F16/248 ,  G06F16/951 ,  G06F16/242 ,  G06F16/2455 ,  G10L15/02 ,  G10L15/187 ,  G06V20/10 ,  G06V40/20 ,  G06F40/295 ,  H04L51/52 ,  H04L51/216 ,  H04L67/50 ,  H04L67/5651 ,  H04L51/18 ,  H04L67/75 ,  G06F16/903 ,  G06F40/205 ,  G06F18/2411 ,  G06N7/01 ,  G06V10/764

CPC classification number: G06V10/82 ,  G06F9/453 ,  G10L15/1815 ,  G10L15/183 ,  G10L15/22 ,  G06F16/338 ,  G06F16/3323 ,  G06F16/3344 ,  G06N20/00 ,  G06F16/9535 ,  G06Q50/01 ,  H04L67/306 ,  G06F16/176 ,  G10L15/063 ,  G10L15/16 ,  G06F3/011 ,  G06F3/017 ,  G06F16/90332 ,  G06F16/24575 ,  H04L51/02 ,  G06F3/167 ,  G10L15/1822 ,  G06N3/08 ,  G10L15/26 ,  G06F16/9038 ,  G06F16/904 ,  G06F40/30 ,  G06F40/40 ,  G06F16/2255 ,  G06F16/2365 ,  G06F7/14 ,  H04L43/0882 ,  H04L43/0894 ,  H04L12/2816 ,  H04L41/20 ,  H04L41/22 ,  H04W12/08 ,  G10L15/07 ,  G10L17/22 ,  G06F16/3329 ,  G06N3/006 ,  G10L17/06 ,  G06F16/24578 ,  G06F16/248 ,  G06F16/951 ,  G06F16/243 ,  G06F16/24552 ,  G06F3/013 ,  G10L15/02 ,  G10L15/187 ,  G06V20/10 ,  G06V40/28 ,  G06F40/295 ,  H04L51/52 ,  H04L51/216 ,  H04L67/535 ,  H04L67/5651 ,  H04L51/18 ,  H04L67/75 ,  G06F16/90335 ,  G06F40/205 ,  G06F18/2411 ,  G06N7/01 ,  G06V10/764 ,  H04L67/10

Abstract: In one embodiment, a method includes receiving a user request from a first user from a client system associated with a first user, wherein the user request comprise a gesture-input from the first user and a speech-input from the first user, determining an intent corresponding to the user request based on the gesture-input by a personalized gesture-classification model associated with the first user, executing one or more tasks based on the determined intent and the speech-input, and sending instructions for presenting execution results of the one or more tasks to the client system responsive the user request.

公开(公告)号：US12299755B2

公开(公告)日：2025-05-13

申请号：US18462101

申请日：2023-09-06

Applicant: Meta Platforms Technologies, LLC

Inventor： Xiaohu Liu ,  Kai Jun Weng ,  Zhiguang Wang ,  Paul Anthony Crook

IPC: G06V40/20 ,  G06F3/01 ,  G06F9/451 ,  G06F9/48 ,  G06F9/54 ,  G06F16/3329 ,  G06F16/9032 ,  G06F16/9536 ,  G06F18/2321 ,  G06F40/205 ,  G06F40/242 ,  G06F40/253 ,  G06F40/295 ,  G06F40/30 ,  G06F40/35 ,  G06F40/56 ,  G06N3/04 ,  G06N3/045 ,  G06N3/047 ,  G06N3/08 ,  G06N20/00 ,  G06Q10/109 ,  G06Q30/0601 ,  G06Q50/00 ,  G06V10/20 ,  G06V10/764 ,  G06V10/82 ,  G06V20/00 ,  G06V20/20 ,  G06V20/30 ,  G06V40/16 ,  G10L15/06 ,  G10L15/08 ,  G10L15/16 ,  G10L15/18 ,  G10L15/22 ,  G10L15/30 ,  G10L15/32 ,  H04L51/18 ,  H04L51/212 ,  H04L51/222 ,  H04L51/224 ,  H04L51/52 ,  H04L67/306 ,  H04L67/75 ,  H04N7/14 ,  G06F3/16 ,  G06V20/40

Abstract: In one embodiment, a method includes maintaining contextual information from a first dialog session, wherein the contextual information comprises identifiers of entities and slots, receiving a user request during the first dialog session, determining a context carryover is required for an agent to execute a task associated with the user request, determining a first subset of the identifiers of the entities and a second subset of the slots of the contextual information from the first dialog session are shareable with the agent based on the agent and first resources of the identifiers of the entities and second resources of the slots, and executing the task by the agent based on the first subset of the identifiers of the entities and the second subset of the slots of the shareable contextual information.

公开(公告)号：US20250037212A1

公开(公告)日：2025-01-30

申请号：US18883896

申请日：2024-09-12

Applicant: Meta Platforms Technologies, LLC

Inventor： Xiaohu Liu ,  Baiyang Liu ,  Rajen Subba

IPC: G06Q50/00 ,  G06F3/01 ,  G06F3/16 ,  G06F9/451 ,  G06F9/48 ,  G06F9/54 ,  G06F16/332 ,  G06F16/9032 ,  G06F16/9536 ,  G06F18/2321 ,  G06F40/205 ,  G06F40/242 ,  G06F40/253 ,  G06F40/295 ,  G06F40/30 ,  G06F40/35 ,  G06F40/56 ,  G06N3/04 ,  G06N3/045 ,  G06N3/047 ,  G06N3/08 ,  G06N20/00 ,  G06Q10/109 ,  G06Q30/0601 ,  G06V10/20 ,  G06V10/764 ,  G06V10/82 ,  G06V20/00 ,  G06V20/20 ,  G06V20/30 ,  G06V20/40 ,  G06V40/16 ,  G06V40/20 ,  G10L15/06 ,  G10L15/08 ,  G10L15/16 ,  G10L15/18 ,  G10L15/22 ,  G10L15/30 ,  G10L15/32 ,  H04L51/18 ,  H04L51/212 ,  H04L51/222 ,  H04L51/224 ,  H04L51/52 ,  H04L67/306 ,  H04L67/75 ,  H04N7/14

Abstract: In one embodiment, a method includes establishing a video call between a plurality of client systems, wherein access to an assistant system is persistently maintained during the video call, receiving, from a first client system of the plurality of client systems, a request by a first user to be performed by the assistant system during the video call, wherein the request references one or more activities associated with one or more users associated with the plurality of client systems, analyzing, by a context engine of the assistant system, images of a scene of the video call to identify the one or more activities within the scene, instructing the assistant system to execute the request based on the identified one or more activities, and sending, to one or more of the plurality of client systems, a response to the request while maintaining the video call between the plurality of client systems.

公开(公告)号：US12125272B2

公开(公告)日：2024-10-22

申请号：US18449525

申请日：2023-08-14

Applicant: Meta Platforms Technologies, LLC

Inventor： Paul Anthony Crook ,  Francislav P. Penov ,  Rajen Subba ,  Xiaohu Liu

IPC: G06V10/82 ,  G06F3/01 ,  G06F3/16 ,  G06F7/14 ,  G06F9/451 ,  G06F16/176 ,  G06F16/22 ,  G06F16/23 ,  G06F16/242 ,  G06F16/2455 ,  G06F16/2457 ,  G06F16/248 ,  G06F16/33 ,  G06F16/332 ,  G06F16/338 ,  G06F16/903 ,  G06F16/9032 ,  G06F16/9038 ,  G06F16/904 ,  G06F16/951 ,  G06F16/9535 ,  G06F18/2411 ,  G06F40/205 ,  G06F40/295 ,  G06F40/30 ,  G06F40/40 ,  G06N3/006 ,  G06N3/08 ,  G06N7/01 ,  G06N20/00 ,  G06Q50/00 ,  G06V10/764 ,  G06V20/10 ,  G06V40/20 ,  G10L15/02 ,  G10L15/06 ,  G10L15/07 ,  G10L15/16 ,  G10L15/18 ,  G10L15/183 ,  G10L15/187 ,  G10L15/22 ,  G10L15/26 ,  G10L17/06 ,  G10L17/22 ,  H04L5/02 ,  H04L12/28 ,  H04L41/00 ,  H04L41/22 ,  H04L43/0882 ,  H04L43/0894 ,  H04L51/02 ,  H04L51/18 ,  H04L51/216 ,  H04L51/52 ,  H04L67/306 ,  H04L67/50 ,  H04L67/5651 ,  H04L67/75 ,  H04W12/08 ,  G10L13/00 ,  G10L13/04 ,  H04L51/046 ,  H04L67/10 ,  H04L67/53

CPC classification number: G06V10/82 ,  G06F3/011 ,  G06F3/013 ,  G06F3/017 ,  G06F3/167 ,  G06F7/14 ,  G06F9/453 ,  G06F16/176 ,  G06F16/2255 ,  G06F16/2365 ,  G06F16/243 ,  G06F16/24552 ,  G06F16/24575 ,  G06F16/24578 ,  G06F16/248 ,  G06F16/3323 ,  G06F16/3329 ,  G06F16/3344 ,  G06F16/338 ,  G06F16/90332 ,  G06F16/90335 ,  G06F16/9038 ,  G06F16/904 ,  G06F16/951 ,  G06F16/9535 ,  G06F18/2411 ,  G06F40/205 ,  G06F40/295 ,  G06F40/30 ,  G06F40/40 ,  G06N3/006 ,  G06N3/08 ,  G06N7/01 ,  G06N20/00 ,  G06Q50/01 ,  G06V10/764 ,  G06V20/10 ,  G06V40/28 ,  G10L15/02 ,  G10L15/063 ,  G10L15/07 ,  G10L15/16 ,  G10L15/1815 ,  G10L15/1822 ,  G10L15/183 ,  G10L15/187 ,  G10L15/22 ,  G10L15/26 ,  G10L17/06 ,  G10L17/22 ,  H04L5/02 ,  H04L12/2816 ,  H04L41/20 ,  H04L41/22 ,  H04L43/0882 ,  H04L43/0894 ,  H04L51/02 ,  H04L51/18 ,  H04L51/216 ,  H04L51/52 ,  H04L67/306 ,  H04L67/535 ,  H04L67/5651 ,  H04L67/75 ,  H04W12/08 ,  G06F2216/13 ,  G10L13/00 ,  G10L13/04 ,  G10L2015/223 ,  G10L2015/225 ,  H04L51/046 ,  H04L67/10 ,  H04L67/53

Abstract: In one embodiment, a method includes receiving a user request from a first user from a client system associated with a first user, wherein the user request comprise a gesture-input from the first user and a speech-input from the first user, determining an intent corresponding to the user request based on the gesture-input by a personalized gesture-classification model associated with the first user, executing one or more tasks based on the determined intent and the speech-input, and sending instructions for presenting execution results of the one or more tasks to the client system responsive the user request.

公开(公告)号：US20230418875A1

公开(公告)日：2023-12-28

申请号：US18462101

申请日：2023-09-06

Applicant: Meta Platforms Technologies, LLC

Inventor： Xiaohu Liu ,  Kai Jun Weng ,  Zhiguang Wang ,  Paul Anthony Crook

IPC: G06F16/9032 ,  G06F9/48 ,  G10L15/22

CPC classification number: G06F16/90332 ,  G06F9/4881 ,  G10L2015/228 ,  G06F9/4862 ,  G10L15/22

Abstract: In one embodiment, a method includes maintaining contextual information from a first dialog session, wherein the contextual information comprises identifiers of entities and slots, receiving a user request during the first dialog session, determining a context carryover is required for an agent to execute a task associated with the user request, determining a first subset of the identifiers of the entities and a second subset of the slots of the contextual information from the first dialog session are shareable with the agent based on the agent and first resources of the identifiers of the entities and second resources of the slots, and executing the task by the agent based on the first subset of the identifiers of the entities and the second subset of the slots of the shareable contextual information.

公开(公告)号：US11699194B2

公开(公告)日：2023-07-11

申请号：US16742668

申请日：2020-01-14

Applicant: Meta Platforms Technologies, LLC

Inventor： Xiaohu Liu ,  Zhiguang Wang ,  Rajen Subba

IPC: G06F40/30 ,  G06F9/54 ,  G06F40/205 ,  G06F40/242 ,  G06N3/04 ,  G06F16/9536 ,  G10L15/18 ,  G10L15/22 ,  G10L15/30 ,  G10L15/32 ,  G06F40/253 ,  G06N20/00 ,  G06F3/01 ,  G06F16/9032 ,  G06F9/48 ,  G10L15/08 ,  H04N7/14 ,  H04L67/306 ,  G06V10/20 ,  G06V20/20 ,  G06V20/30 ,  G06V20/40 ,  G06V40/16 ,  G06Q50/00 ,  H04L51/52 ,  H04L51/212 ,  H04L67/75 ,  G06F9/451 ,  G06F16/332 ,  G06F40/35 ,  G06K9/62 ,  G06F3/16 ,  G06N3/08 ,  G10L15/06 ,  G10L15/16 ,  G06F18/2321 ,  G06N3/045 ,  G06N3/047 ,  G06V10/764 ,  G06V10/82 ,  G06F40/56 ,  H04L51/18 ,  G06V20/00 ,  G06V40/20 ,  H04L51/222 ,  H04L51/224 ,  G06Q10/109

CPC classification number: G06Q50/01 ,  G06F3/011 ,  G06F3/013 ,  G06F9/453 ,  G06F9/485 ,  G06F9/4881 ,  G06F9/547 ,  G06F16/3329 ,  G06F16/90332 ,  G06F16/9536 ,  G06F18/2321 ,  G06F40/205 ,  G06F40/242 ,  G06F40/253 ,  G06F40/30 ,  G06F40/35 ,  G06F40/56 ,  G06N3/045 ,  G06N3/047 ,  G06N3/0454 ,  G06N3/0472 ,  G06N3/08 ,  G06N20/00 ,  G06Q10/109 ,  G06V10/255 ,  G06V10/764 ,  G06V10/82 ,  G06V20/00 ,  G06V20/20 ,  G06V20/30 ,  G06V40/16 ,  G06V40/25 ,  G10L15/063 ,  G10L15/08 ,  G10L15/16 ,  G10L15/1815 ,  G10L15/1822 ,  G10L15/22 ,  G10L15/30 ,  G10L15/32 ,  H04L51/18 ,  H04L51/212 ,  H04L51/222 ,  H04L51/224 ,  H04L51/52 ,  H04L67/306 ,  H04L67/75 ,  H04N7/147 ,  G06F3/017 ,  G06F3/167 ,  G06V20/41 ,  G06V40/174 ,  G06V2201/10 ,  G10L2015/088 ,  G10L2015/223 ,  G10L2015/227

Abstract: In one embodiment, a method includes receiving a first user request to execute a first task comprising one or more sub-tasks from a client system associated with a user, receiving a second user request to suspend the first task from the client system, storing a dialog state for the first task in response to the second user request, wherein the dialog state comprises results of executed sub-tasks and a status indicating the first task is suspended, receiving a third user request to resume the first task, accessing the dialog state for the first task, and providing a prompt to resume the first task at the client system.