公开(公告)号：US20240203143A1

公开(公告)日：2024-06-20

申请号：US18454459

申请日：2023-08-23

Applicant: Samsung Electronics Co., Ltd.

Inventor： Lingyu Zhang ,  Ting Hua ,  Yilin Shen ,  Hongxia Jin

IPC: G06V20/70 ,  G06F40/284 ,  G06V10/774

CPC classification number: G06V20/70 ,  G06F40/284 ,  G06V10/774

Abstract: A method includes obtaining an image, a set of attribute labels, and a set of object labels and performing prompt tuning of a pre-trained vision-language model having first and second textual encoders and a vision encoder. The model is trained during prompt tuning to select one attribute label and one object label that match content in the image. Performing the prompt tuning includes, for each attribute label-object label pair, generating object textual features associated with the object label using the first textual encoder, generating attribute textual features associated with the attribute label using the second textual encoder, and generating image features associated with the image using the vision encoder. Intermediate outputs from initial layers of the textual encoders and the vision encoder are combined to generate layer-specific learnable prompt tokens that are appended to inputs of specified layers in the first and second textual encoders and the vision encoder.

公开(公告)号：US20240185850A1

公开(公告)日：2024-06-06

申请号：US18352601

申请日：2023-07-14

Applicant: Samsung Electronics Co., Ltd.

Inventor： Rakshith Sharma Srinivasa ,  Yashas Malur Saidutta ,  Ching-Hua Lee ,  Chou-Chang Yang ,  Yilin Shen ,  Hongxia Jin

IPC: G10L15/22 ,  G10L15/02 ,  G10L15/06 ,  G10L15/18 ,  G10L25/78

CPC classification number: G10L15/22 ,  G10L15/02 ,  G10L15/063 ,  G10L15/18 ,  G10L25/78 ,  G10L2015/088 ,  G10L2015/223

Abstract: A method includes extracting, using a keyword detection model, audio features from audio data. The method also includes processing the audio features by a first layer of the keyword detection model configured to predict a first likelihood that the audio data includes speech. The method also includes processing the audio features by a second layer of the keyword detection model configured to predict a second likelihood that the audio data includes keyword-like speech. The method also includes processing the audio features by a third layer of the keyword detection model configured to predict a third likelihood, for each of a plurality of possible keywords, that the audio data includes the keyword. The method also includes identifying a keyword included in the audio data. The method also includes generating instructions to perform an action based at least in part on the identified keyword.

公开(公告)号：US20240104309A1

公开(公告)日：2024-03-28

申请号：US18465648

申请日：2023-09-12

Applicant: Samsung Electronics Co., Ltd.

Inventor： Yen-Chang Hsu ,  Harshavardhan Kamarthi ,  Yilin Shen ,  Hongxia Jin

IPC: G06F40/35 ,  G06F40/166 ,  G06F40/284 ,  G06F40/40 ,  G06N3/09

CPC classification number: G06F40/35 ,  G06F40/166 ,  G06F40/284 ,  G06F40/40 ,  G06N3/09

Abstract: A method includes receiving an input for a large language model (LLM) from a user. The method also includes generating one or more token embeddings based on the input. The method further includes generating one or more prompt embeddings based on the input using a contextual prompt generator (CPG), the one or more prompt embeddings representing new or updated information that is not contained in existing knowledge of the LLM. The method also includes providing the one or more token embeddings and the one or more prompt embeddings to the LLM. In addition, the method includes outputting a prediction based on the one or more token embeddings and the one or more prompt embeddings using the LLM, wherein the prediction reflects the new or updated information represented by the one or more prompt embeddings.

公开(公告)号：US20240080423A1

公开(公告)日：2024-03-07

申请号：US18057126

申请日：2022-11-18

Applicant: Samsung Electronics Co., Ltd.

Inventor： Wenbo Li ,  Zhipeng Mo ,  Yi Wei ,  Burak Uzkent ,  Qian Lou ,  Yilin Shen ,  Hongxia Jin

IPC: H04N9/64

CPC classification number: H04N9/64

Abstract: A method includes obtaining raw image data, where the raw image data includes data values each having most significant bits and least significant bits. The method also includes providing the raw image data to a trained machine learning model and generating processed image data using the trained machine learning model. The method further includes presenting an image based on the processed image data. The trained machine learning model is trained to modulate a feature map associated with the most significant bits of the data values of the raw image data based on the least significant bits of the data values of the raw image data in order to generate a fusion of the most significant bits and the least significant bits of the data values of the raw image data.

公开(公告)号：US11775815B2

公开(公告)日：2023-10-03

申请号：US16535380

申请日：2019-08-08

Applicant: Samsung Electronics Co., Ltd.

Inventor： Yilin Shen ,  Yue Deng ,  Avik Ray ,  Hongxia Jin

IPC: G06N3/08 ,  G06N5/04

CPC classification number: G06N3/08 ,  G06N5/04

Abstract: An electronic device including a deep memory model includes at least one memory and at least one processor coupled to the at least one memory. The at least one processor is configured to receive input data to the deep memory model. The at least one processor is also configured to extract a history state of an external memory coupled to the deep memory model based on the input data. The at least one processor is further configured to update the history state of the external memory based on the input data. In addition, the at least one processor is configured to output a prediction based on the extracted history state of the external memory.

公开(公告)号：US11721090B2

公开(公告)日：2023-08-08

申请号：US16041479

申请日：2018-07-20

Applicant: Samsung Electronics Co., Ltd.

Inventor： Yue Deng ,  Yilin Shen ,  Hongxia Jin

IPC: G06V10/82 ,  G06N3/08 ,  G06V20/40 ,  H04N21/25 ,  H04N21/466 ,  H04N21/45 ,  G06F18/241 ,  G06F18/245 ,  G06F18/214 ,  G06N3/045 ,  G06N3/047 ,  G06V10/764 ,  G06V10/774 ,  G06F16/435 ,  G06Q30/0251

CPC classification number: G06V10/82 ,  G06F16/435 ,  G06F18/2155 ,  G06F18/241 ,  G06F18/245 ,  G06N3/045 ,  G06N3/047 ,  G06N3/08 ,  G06Q30/0269 ,  G06V10/764 ,  G06V10/7753 ,  G06V20/46 ,  H04N21/251 ,  H04N21/45 ,  H04N21/466 ,  H04N21/4666 ,  H04N21/4668

Abstract: A recommendation method includes retrieving content consumption data including content consumed and content not consumed. Based on the content consumption data, identifying a first piece of content not consumed. A first feature of the first piece of content related to negative consumption of the first piece of content is determined. A first system is used to revise the first feature to a second feature. A second piece of content including the second feature is provided to an electronic device. The second piece of content is a revised instance of the first piece of content.

公开(公告)号：US20230245435A1

公开(公告)日：2023-08-03

申请号：US17589535

申请日：2022-01-31

Applicant: Samsung Electronics Co., Ltd.

Inventor： Changsheng Zhao ,  Burak Uzkent ,  Yilin Shen ,  Hongxia Jin

IPC: G06V10/80 ,  G06V10/778 ,  G06V10/774 ,  G06F40/279

CPC classification number: G06V10/811 ,  G06V10/778 ,  G06V10/774 ,  G06F40/279

Abstract: A method includes obtaining a batch of training data including multiple paired image-text pairs and multiple unpaired image-text pairs, where each paired image-text pair and each unpaired image-text pair includes an image and a text. The method also includes training a machine learning model using the training data based on an optimization of a combination of losses. The losses include, for each paired image-text pair, (i) a first multi-modal representation loss based on the paired image-text pair and (ii) a second multi-modal representation loss based on two or more unpaired image-text pairs, selected from among the multiple unpaired image-text pairs, wherein each of the two or more unpaired image-text pairs includes either the image or the text of the paired image-text pair.

公开(公告)号：US11455471B2

公开(公告)日：2022-09-27

申请号：US16947258

申请日：2020-07-24

Applicant: Samsung Electronics Co., Ltd.

Inventor： Yilin Shen ,  Hongxia Jin

IPC: G06F40/30 ,  G06N3/08 ,  G06F40/279

Abstract: A method includes obtaining, using at least one processor of an electronic device, a base natural language understanding (NLU) model that includes a word embedding layer, where the word embedding layer is associated with at least one training utterance. The method also includes calculating, using the at least one processor, a regularization loss value for use in a determination of an intent detection loss, where the regularization loss value reveals an effect of word embeddings on intent determination of the training utterance. The method further includes retraining, using the at least one processor, the word embedding layer of the base NLU model using the intent detection loss to obtain a retrained NLU model.

公开(公告)号：US11275896B2

公开(公告)日：2022-03-15

申请号：US15986633

申请日：2018-05-22

Applicant: Samsung Electronics Co., Ltd.

Inventor： Avik Ray ,  Yilin Shen ,  Hongxia Jin

IPC: G10L15/22 ,  G06F40/30 ,  G06F9/451 ,  G06N5/02 ,  G06F40/205 ,  G06F40/253 ,  G10L15/07 ,  G10L15/06

Abstract: A method includes determining, by an electronic device, a skill from a first natural language (NL) input. Upon successful determination of the skill, the first NL input is transmitted to a custom skill parser for determination of a skill intent. The custom skill parser is trained based on data including at least a custom training data set. Upon unsuccessful determination of the skill, the first NL input is transmitted to a generic parser for determination of a general intent of the first NL input.

公开(公告)号：US20210383272A1

公开(公告)日：2021-12-09

申请号：US17166908

申请日：2021-02-03

Applicant: Samsung Electronics Co., Ltd.

Inventor： Ting Hua ,  Yilin Shen ,  Changsheng Zhao ,  Hongxia Jin

IPC: G06N20/00 ,  G06K9/62

Abstract: A continual learning method includes obtaining an input data including a trained model, continual learning (CL) Information, and training data by an electronic device. The method also includes re-training, using the electronic device, the model for a task based on the training data. The method also includes updating, using the electronic device, the CL Information based on the model and the training data. The method further includes selecting a first set of exemplars from the training data based on data associated with the CL Information. The CL Information includes a first group of variables associated with the model and a second group of variables associated with the model that changes to the first group of variables have stronger impact to the model's performance of the task than changes to the second group of variables.