公开(公告)号：US12086698B2

公开(公告)日：2024-09-10

申请号：US17484618

申请日：2021-09-24

Applicant: Salesforce, Inc.

Inventor： Mingfei Gao ,  Zeyuan Chen ,  Ran Xu

IPC: G06N20/20 ,  G06N3/084 ,  G06N5/01 ,  G06N5/04 ,  G06V30/412 ,  G06V30/413

CPC classification number: G06N20/20 ,  G06N3/084 ,  G06N5/01 ,  G06N5/04 ,  G06V30/412 ,  G06V30/413

Abstract: A field extraction system that does not require field-level annotations for training is provided. Specifically, the training process is bootstrapped by mining pseudo-labels from unlabeled forms using simple rules. Then, a transformer-based structure is used to model interactions between text tokens in the input form and predict a field tag for each token accordingly. The pseudo-labels are used to supervise the transformer training. As the pseudo-labels are noisy, a refinement module that contains a sequence of branches is used to refine the pseudo-labels. Each of the refinement branches conducts field tagging and generates refined labels. At each stage, a branch is optimized by the labels ensembled from all previous branches to reduce label noise.

公开(公告)号：US20250053793A1

公开(公告)日：2025-02-13

申请号：US18494393

申请日：2023-10-25

Applicant: Salesforce, Inc.

Inventor： Zhiwei Liu ,  Weiran Yao ,  Jianguo Zhang ,  Le Xue ,  Shelby Heinecke ,  Rithesh Murthy ,  Yihao Feng ,  Zeyuan Chen ,  Juan Carlos Niebles Duque ,  Devansh Arpit ,  Ran Xu ,  Lik Mui ,  Huan Wang ,  Caiming Xiong ,  Silvio Savarese

IPC: G06N3/047 ,  G06N3/092

Abstract: Embodiments described herein provide a method of predicting an action by a plurality of language model augmented agents (LAAs). In at least one embodiment, a controller receives a task instruction to be performed using an environment. The controller receives an observation of a first state from the environment. The controller selects a LAA from the plurality of LAAs based on the task instruction and the observation. The controller obtains an output from the selected LAA generated using an input combining the task instruction, the observation, and an LAA-specific prompt template. The controller determines the action based on the output. The controller causes the action to be performed on the environment thereby causing the first state of the environment to change to a second state.

公开(公告)号：US20240104809A1

公开(公告)日：2024-03-28

申请号：US18161680

申请日：2023-01-30

Applicant: Salesforce, Inc.

Inventor： Ning Yu ,  Chia-Chih Chen ,  Zeyuan Chen ,  Caiming Xiong ,  Juan Carlos Niebles Duque ,  Ran Xu ,  Rui Meng

IPC: G06T11/60 ,  G06F40/106 ,  G06F40/126 ,  G06N20/00 ,  G06T9/00

CPC classification number: G06T11/60 ,  G06F40/106 ,  G06F40/126 ,  G06N20/00 ,  G06T9/00 ,  G06T2200/24 ,  G06T2210/12

Abstract: Embodiments described herein provide systems and methods for multimodal layout generations for digital publications. The system may receive as inputs, a background image, one or more foreground texts, and one or more foreground images. Feature representations of the background image may be generated. The foreground inputs may be input to a layout generator which has cross attention to the background image feature representations in order to generate a layout comprising of bounding box parameters for each input item. A composite layout may be generated based on the inputs and generated bounding boxes. The resulting composite layout may then be displayed on a user interface.

公开(公告)号：US12235850B2

公开(公告)日：2025-02-25

申请号：US17588022

申请日：2022-01-28

Applicant: Salesforce, Inc.

Inventor： Luyu Yang ,  Mingfei Gao ,  Zeyuan Chen ,  Ran Xu ,  Chetan Ramaiah

IPC: G06F16/2455 ,  G06F16/242 ,  G06N20/00

Abstract: Embodiments described herein provide an online domain adaptation framework based on cross-domain bootstrapping for online domain adaptation, in which the target domain streaming data is deleted immediately after adapted. At each online query, the data diversity is increased across domains by bootstrapping the source domain to form diverse combinations with the current target query. To fully take advantage of the valuable discrepancies among the diverse combinations, a set of independent learners are trained to preserve the differences. The knowledge of the learners is then integrated by exchanging their predicted pseudo-labels on the current target query to co-supervise the learning on the target domain, but without sharing the weights to maintain the learners' divergence.

公开(公告)号：US20250045567A1

公开(公告)日：2025-02-06

申请号：US18498257

申请日：2023-10-31

Applicant: Salesforce, Inc.

Inventor： Weiran Yao ,  Shelby Heinecke ,  Juan Carlos Niebles Duque ,  Zhiwei Liu ,  Yihao Feng ,  Le Xue ,  Rithesh Murthy ,  Zeyuan Chen ,  Jianguo Zhang ,  Devansh Arpit ,  Ran Xu ,  Lik Mui ,  Huan Wang ,  Caiming Xiong ,  Silvio Savarese

IPC: G06N3/0455 ,  G06N3/092

Abstract: Embodiments described herein provide for optimizing a language model (LM) agent. In at least one embodiment, and LM agent comprises an “actor” LM and a “retrospective LM which provides reflections on attempts by the actor LM. The reflections are used to update subsequent prompts to the actor LM. Optimizing the LM agent comprises fine-tuning parameters of the retrospective LM while keeping parameters of the actor LM frozen. A gradient may be determined by a change in reward from the environment based on actions taken by the actor LM with and without a reflection of the retrospective LM. Using this gradient, parameters of the retrospective LM may be updated via backpropagation.

公开(公告)号：US20240054350A1

公开(公告)日：2024-02-15

申请号：US18064122

申请日：2022-12-09

Applicant: Salesforce Inc.

Inventor： Yutong Dai ,  Zeyuan Chen ,  Junnan Li

IPC: G06N3/098

CPC classification number: G06N3/098

Abstract: Embodiments described herein provide systems and methods for federated learning. A central system may store a neural network model which has a body of a number of layers, and a classification layer comprising class prototypes which classifies the latent representations output by the body of the model. The central system may initialize the class prototypes so that they are uniformly distributed in the representation space. The model and class prototypes may be broadcast to a number of client systems, which update the body of the model locally while keeping the class prototypes fixed. The clients may return information to the central system including updated local model parameters, and a local representation of the classes based on the latent representation of items in the local training data. Based on the information from the clients, the neural network model may be updated. This process may be repeated iteratively.