公开(公告)号：US11886461B2

公开(公告)日：2024-01-30

申请号：US16528175

申请日：2019-07-31

Applicant: Salesforce, Inc.

Inventor： Arun Kumar Jagota ,  Stanislav Georgiev

IPC: G06N20/00 ,  G06N7/01 ,  G06F16/25 ,  G06F16/2455

CPC classification number: G06F16/258 ,  G06F16/2456 ,  G06N7/01 ,  G06N20/00

Abstract: A system tokenizes raw values and corresponding standardized values into raw token sequences and corresponding standardized token sequences. A machine-learning model learns standardization from token insertions and token substitutions that modify the raw token sequences to match the corresponding standardized token sequences. The system tokenizes an input value into an input token sequence. The machine-learning model determines a probability of inserting an insertion token after an insertion markable token in the input token sequence. If the probability of inserting the insertion token satisfies a threshold, the system inserts the insertion token after the insertion markable token in the input token sequence. The machine-learning model determines a probability of substituting a substitution token for a substitutable token in the input token sequence. If the probability of substituting the substitution token satisfies another threshold, the system substitutes the substitution token for the substitutable token in the input token sequence.