Pretraining a language machine-learning model

    公开(公告)号:US12236206B1

    公开(公告)日:2025-02-25

    申请号:US17331478

    申请日:2021-05-26

    Abstract: In one embodiment, a method includes accessing a first document, accessing a plurality of second documents, calculating a relevance score for each of the plurality of second documents indicating a degree of relevance of the second document to the first document using an encoder of a machine-learning model, selecting a subset of the second documents based on their corresponding relevance scores, generating a target document by using the machine-learning model to process the subset of second documents and their corresponding relevance scores, and updating parameters of the machine-learning model based on a comparison between the first document and the generated target document.

    Physical examination information distribution to distribution objects with workload information and matching model

    公开(公告)号:US12217838B2

    公开(公告)日:2025-02-04

    申请号:US17440219

    申请日:2020-12-25

    Inventor: Zhenzhong Zhang

    Abstract: There are provided a method and an apparatus for distributing physical examination information, an electronic device, a computer-readable storage medium, and a computer program product. The method includes: obtaining physical examination information and information of a plurality of distribution objects; inputting the physical examination information and the information of the plurality of distribution objects into an information matching model obtained by pre-training to obtain a matching degree between the physical examination information and the plurality of distribution objects; and determining a target distribution object from the plurality of distribution objects according to the matching degree between the physical examination information and each of the plurality of distribution objects, and distributing the physical examination information to the target distribution object.

    Method and system for probably robust classification with detection of adversarial examples

    公开(公告)号:US12210966B2

    公开(公告)日:2025-01-28

    申请号:US17035203

    申请日:2020-09-28

    Abstract: A computer-implemented method for training a machine-learning network includes receiving an input data from a sensor, wherein the input data includes a perturbation, wherein the input data is indicative of image, radar, sonar, or sound information, obtain a worst-case bound on a classification error and loss for perturbed versions of the input data, utilizing at least bounding of one or more hidden layer values, in response to the input data, train a classifier, wherein the classifier includes a plurality of classes, including an additional abstain class, wherein the abstain class is determined in response to at least bounding the input data, outputting a classification in response to the input data, and output a trained classifier configured to detect the additional abstain class in response to the input data classifier with a plurality of classes, including an additional abstain class.

    Trajectory prediction using directed graph and destination features

    公开(公告)号:US12198460B2

    公开(公告)日:2025-01-14

    申请号:US17635792

    申请日:2019-09-16

    Abstract: Systems, methods, apparatuses, and computer program products to provide stochastic trajectory prediction using social graph networks. An operation may comprise determining a first feature vector describing destination features of a first person depicted in an image, generating a directed graph for the image based on all people depicted in the image, determining, for the first person, a second feature vector based on the directed graph and the destination features, sampling a value of a latent variable from a learned prior distribution, the latent variable to correspond to a first time interval, and generating, based on the sampled value and the feature vectors by a hierarchical long short-term memory (LSTM) executing on a processor, an output vector comprising a direction of movement and a speed of the direction of movement of the first person at a second time interval, subsequent to the first time interval.

    MACHINE LEARNING MODEL WITH DEPTH PROCESSING UNITS

    公开(公告)号:US20250005339A1

    公开(公告)日:2025-01-02

    申请号:US18829480

    申请日:2024-09-10

    Abstract: Representative embodiments disclose machine learning classifiers used in scenarios such as speech recognition, image captioning, machine translation, or other sequence-to-sequence embodiments. The machine learning classifiers have a plurality of time layers, each layer having a time processing block and a depth processing block. The time processing block is a recurrent neural network such as a Long Short Term Memory (LSTM) network. The depth processing blocks can be an LSTM network, a gated Deep Neural Network (DNN) or a maxout DNN. The depth processing blocks account for the hidden states of each time layer and uses summarized layer information for final input signal feature classification. An attention layer can also be used between the top depth processing block and the output layer.

    Interleaver design and pairwise codeword distance distribution enhancement for turbo autoencoder

    公开(公告)号:US12175353B2

    公开(公告)日:2024-12-24

    申请号:US17726506

    申请日:2022-04-21

    Abstract: A symmetric interleaver for a Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) encoder and a circular padding mode are disclosed. The interleaver interleaves elements of an input block to form an output block in which an output neighborhood of elements for each element of the output block is symmetric to an input neighborhood of elements for each element of the input block. A position of an element of the input block is interleaved based on an index i of the position times a parameter δ modulo K in which the parameter δ is relatively prime with K. A test loss function may be used to train the encoder that includes a Binary Cross Entropy (BCE) loss function plus a function that minimizes a number of codeword pairs based on a Euclidean distance. The RNN encoder may be implemented as part of a Turbo Autoencoder (TurboAE) encoder.

    TRAINING OPTIMIZATION FOR LOW MEMORY FOOTPRINT

    公开(公告)号:US20240419973A1

    公开(公告)日:2024-12-19

    申请号:US18742948

    申请日:2024-06-13

    Abstract: A method is described. The method includes profiling a model for a learning network having a plurality of layers and associated memory. The layers include weight layers and activation layers. The plurality of weight layers including weights. The method also includes determining, based on the profiling, a training technique for the model on the learning network. The determination of the training technique includes optimizing a latency for at least one training iteration for a capacity of the associated memory. The training iteration(s) include at least one update of the weights for the weight layers.

Patent Agency Ranking