公开(公告)号：US20240311693A1

公开(公告)日：2024-09-19

申请号：US18592250

申请日：2024-02-29

Applicant: Samsung Electronics Co., Ltd.

Inventor： James S. Smith ,  Yen-Chang Hsu ,  Yilin Shen ,  Hongxia Jin ,  Lingyu Zhang ,  Ting Hua

IPC: G06N20/00

CPC classification number: G06N20/00

Abstract: A method includes obtaining input data associated with a new concept to be learned by a trained machine learning model. The method also includes identifying initial weights of the trained machine learning model and one or more previous weight deltas associated with the trained machine learning model. The method further includes identifying one or more additional weight deltas based on the input data and guided by the initial weights and the one or more previous weight deltas. In addition, the method includes integrating the one or more additional weight deltas into the trained machine learning model. The one or more additional weight deltas are integrated into the trained machine learning model by identifying updated weights for the trained machine learning model based on the initial weights, the one or more previous weight deltas, and the one or more additional weight deltas.

公开(公告)号：US20250021826A1

公开(公告)日：2025-01-16

申请号：US18629162

申请日：2024-04-08

Applicant: Samsung Electronics Co., Ltd.

Inventor： Shangqian Gao ,  Ting Hua ,  Yen-Chang Hsu ,  Yilin Shen ,  Hongxia Jin

IPC: G06N3/0985 ,  G06N3/0495

Abstract: In one embodiment, a method includes accessing at least a portion of a training dataset for a trained neural network that includes multiple layers, where each layer includes a number of parameters, and where the training dataset includes multiple training samples that each include an input and a ground-truth output used to train the trained neural network. The method further includes training a hypernetwork to generate a layer-specific compression mask for each of one or more of the multiple layers of the trained neural network. The method further includes generating, by the trained hypernetwork, a final layer-specific compression mask for the trained neural network and compressing the trained neural network by reducing, for each of the one or more layers of the neural network, the number of parameters of that layer according to the final layer-specific compression mask.

公开(公告)号：US20230177338A1

公开(公告)日：2023-06-08

申请号：US18073383

申请日：2022-12-01

Applicant: Samsung Electronics Co., Ltd.

Inventor： Qian Lou ,  Yen-Chang Hsu ,  Burak Uzkent ,  Ting Hua ,  Yilin Shen ,  Hongxia Jin

IPC: G06N3/082 ,  G06V10/82 ,  G06V10/772

CPC classification number: G06N3/082 ,  G06V10/82 ,  G06V10/772

Abstract: A method includes obtaining, using a first electronic device, a weight matrix associated with a trained transformer model. The method also includes factorizing the weight matrix into a dictionary weight matrix and an intermediate matrix. The method further includes pruning the intermediate matrix to generate a sparse intermediate matrix. The method also includes fine-tuning the sparse intermediate matrix based on a training dataset to generate a fine-tuned sparse intermediate matrix. The method further includes determining an index matrix and a coefficient matrix based on the fine-tuned sparse intermediate matrix. In addition, the method includes deploying the dictionary weight matrix, the index matrix, and the coefficient matrix to a second electronic device without deploying the weight matrix to the second electronic device. A number of parameters in the dictionary weight matrix, the index matrix, and the coefficient matrix is smaller than a number of parameters in the weight matrix.

公开(公告)号：US20250029005A1

公开(公告)日：2025-01-23

申请号：US18669413

申请日：2024-05-20

Applicant: Samsung Electronics Co., Ltd.

Inventor： Ting Hua ,  Xiao Li ,  Shangqian Gao ,  Yen-Chang Hsu ,  Yilin Shen ,  Hongxia Jin

IPC: G06N20/00 ,  G10L15/18 ,  G10L15/26

Abstract: A method includes accessing a plurality of weight matrices of a machine learning model. The method also includes, for each weight matrix, decomposing the weight matrix into a U matrix, an S matrix, and a V matrix using singular value decomposition. The S matrix is a diagonal matrix, and a singular group corresponds to each element in the S matrix. The method further includes, for each weight matrix, determining an importance score of each singular group. The importance score of the singular group represents a change in loss if the singular group is removed from the machine learning model. The method also includes, for each weight matrix, ranking the singular groups across the plurality of weight matrices based on the importance scores. In addition, the method includes, for each weight matrix, identifying one or more of the singular groups to prune based on the ranking of the singular groups.

公开(公告)号：US20240203143A1

公开(公告)日：2024-06-20

申请号：US18454459

申请日：2023-08-23

Applicant: Samsung Electronics Co., Ltd.

Inventor： Lingyu Zhang ,  Ting Hua ,  Yilin Shen ,  Hongxia Jin

IPC: G06V20/70 ,  G06F40/284 ,  G06V10/774

CPC classification number: G06V20/70 ,  G06F40/284 ,  G06V10/774

Abstract: A method includes obtaining an image, a set of attribute labels, and a set of object labels and performing prompt tuning of a pre-trained vision-language model having first and second textual encoders and a vision encoder. The model is trained during prompt tuning to select one attribute label and one object label that match content in the image. Performing the prompt tuning includes, for each attribute label-object label pair, generating object textual features associated with the object label using the first textual encoder, generating attribute textual features associated with the attribute label using the second textual encoder, and generating image features associated with the image using the vision encoder. Intermediate outputs from initial layers of the textual encoders and the vision encoder are combined to generate layer-specific learnable prompt tokens that are appended to inputs of specified layers in the first and second textual encoders and the vision encoder.

公开(公告)号：US20210383272A1

公开(公告)日：2021-12-09

申请号：US17166908

申请日：2021-02-03

Applicant: Samsung Electronics Co., Ltd.

Inventor： Ting Hua ,  Yilin Shen ,  Changsheng Zhao ,  Hongxia Jin

IPC: G06N20/00 ,  G06K9/62

Abstract: A continual learning method includes obtaining an input data including a trained model, continual learning (CL) Information, and training data by an electronic device. The method also includes re-training, using the electronic device, the model for a task based on the training data. The method also includes updating, using the electronic device, the CL Information based on the model and the training data. The method further includes selecting a first set of exemplars from the training data based on data associated with the CL Information. The CL Information includes a first group of variables associated with the model and a second group of variables associated with the model that changes to the first group of variables have stronger impact to the model's performance of the task than changes to the second group of variables.