公开(公告)号：US20240185035A1

公开(公告)日：2024-06-06

申请号：US18162535

申请日：2023-01-31

Applicant: Salesforce, Inc.

Inventor： Ning Yu ,  Can Qin ,  Chen Xing ,  Shu Zhang ,  Stefano Ermon ,  Caiming Xiong ,  Ran Xu

IPC: G06N3/0455 ,  G06T5/00

CPC classification number: G06N3/0455 ,  G06T5/002 ,  G06T2207/20084

Abstract: Embodiments described herein provide a mechanism for replacing existing text encoders in text-to-image generation models with more powerful pre-trained language models. Specifically, a translation network is trained to map features from the pre-trained language model output into the space of the target text encoder. The training preserves the rich structure of the pre-trained language model while allowing it to operate within the text-to-image generation model. The resulting modularized text-to-image model receives prompt and generates an image representing the features contained in the prompt.

公开(公告)号：US20240119308A1

公开(公告)日：2024-04-11

申请号：US18159036

申请日：2023-01-24

Applicant: Salesforce, Inc.

Inventor： Arundhati Banerjee ,  Stephan Zheng ,  Soham Phade ,  Stefano Ermon

IPC: G06N3/0985

CPC classification number: G06N3/0985

Abstract: Embodiments provide a method for predicting agent actions for neural network based agents according to an intervention. The method includes obtaining a first agent action at a first time step and a first intervention generated according to an intervention policy. The method also includes generating, by the neural network based agent model, a predicted agent action conditioned on the first agent action and the first intervention. The method also includes generating, by a neural network based intervention model, a second intervention according to the intervention policy and conditioned on the first agent action, the first intervention, and the predicted agent action. The method further includes executing a second agent action according to an agent policy that incurs a reward based on the second intervention. The method further includes training the neural network based intervention model by updating parameters of the neural network based intervention model based on an expected return.