NORMALIZATION IN DEEP CONVOLUTIONAL NEURAL NETWORKS

    公开(公告)号:US20230237309A1

    公开(公告)日:2023-07-27

    申请号:US18180841

    申请日:2023-03-08

    IPC分类号: G06N3/04 G06N3/08

    CPC分类号: G06N3/04 G06N3/08

    摘要: A device for machine learning is provided, including a first neural network layer, a second neural network layer with a normalization layer arranged in between. The normalization layer is configured to, when the device is undergoing training on a batch of training samples, receive multiple outputs of the first neural network layer for a plurality of training samples of the batch, each output comprising multiple data values for different indices on a first dimension and a second dimension; group the outputs into multiple groups based on the indices on the first and second dimensions; form a normalization output for each group which are provided as input to the second neural network layer. According to the application, the training of a deep convolutional neural network with good performance that performs stably at different batch sizes and is generalizable to multiple vision tasks is achieved, thereby improving the performance of the training.

    Method and apparatus of data processing using multiple types of non-linear combination processing

    公开(公告)号:US11334758B2

    公开(公告)日:2022-05-17

    申请号:US16729043

    申请日:2019-12-27

    IPC分类号: G06K9/62

    摘要: The method includes: obtaining a plurality of pieces of feature data; automatically performing two different types of nonlinear combination processing operations on the plurality of pieces of feature data to obtain two groups of processed data, where the two groups of processed data include a group of higher-order data and a group of lower-order data, the higher-order data is related to a nonlinear combination of m pieces of feature data in the plurality of pieces of feature data, and the lower-order data is related to a nonlinear combination of n pieces of feature data in the plurality of pieces of feature data, where m≥3, and m>n≥2; and determining prediction data based on a plurality of pieces of target data, where the plurality of pieces of target data include the two groups of processed data.

    METHOD AND DEVICE FOR COMPRESSING FLOW DATA
    3.
    发明申请

    公开(公告)号:US20170366197A1

    公开(公告)日:2017-12-21

    申请号:US15697066

    申请日:2017-09-06

    IPC分类号: H03M7/30

    CPC分类号: H03M7/30 H03M7/3064

    摘要: A method for compressing flow data, including: generating multiple line segments according to flow data and a predefined maximum error that are acquired; obtaining a target piecewise linear function according to the multiple line segments, where the target piecewise linear function includes multiple linear functions, and an intersection set of value ranges of independent variables of every two linear functions among the multiple linear functions includes a maximum of one value; and outputting a reference data point according to the target piecewise linear function, where the reference data point includes a point of continuity and a point of discontinuity of the target piecewise linear function. In this way, a maximum error, a target piecewise linear function is further determined according to the multiple line segments, and a point of continuity and a point of discontinuity of the target piecewise linear function are used to represent compressed flow data.

    Deep Learning Training Method for Computing Device and Apparatus

    公开(公告)号:US20230206069A1

    公开(公告)日:2023-06-29

    申请号:US18175936

    申请日:2023-02-28

    IPC分类号: G06N3/08 G06N3/045

    CPC分类号: G06N3/08 G06N3/045

    摘要: A deep learning training method includes obtaining a training set, a first neural network, and a second neural network, where shortcut connections included in the first neural network are less than shortcut connections included in the second neural network; performing at least one time of iterative training on the first neural network based on the training set, to obtain a trained first neural network, where any iterative training includes: using a first output of at least one first intermediate layer in the first neural network as an input of at least one network layer in the second neural network, to obtain an output result of the at least one network layer; and updating the first neural network according to a first loss function.

    Similarity measurement method and device

    公开(公告)号:US10579703B2

    公开(公告)日:2020-03-03

    申请号:US15694559

    申请日:2017-09-01

    摘要: A similarity measurement method includes: obtaining a directional relationship between nodes in a network, and determining a transition matrix; calculating a constraint matrix according to the transition matrix and an obtained attenuation factor; constructing a system of linear equations, where a coefficient matrix of the system of linear equations is the constraint matrix, and a variable of the system of linear equations is a correction vector; solving the system of linear equations by means of iteration by using a Jacobi method, and determining the correction vector; and calculating similarities between the nodes according to the transition matrix, the attenuation factor, and a diagonal correction matrix that is generated according to the correction vector. In the method, the correction vector is determined by using the Jacobi method, and further the similarities between the nodes may be calculated.

    Method and device for compressing flow data

    公开(公告)号:US10218381B2

    公开(公告)日:2019-02-26

    申请号:US15697066

    申请日:2017-09-06

    IPC分类号: H03M7/30

    摘要: A method for compressing flow data, including: generating multiple line segments according to flow data and a predefined maximum error that are acquired; obtaining a target piecewise linear function according to the multiple line segments, where the target piecewise linear function includes multiple linear functions, and an intersection set of value ranges of independent variables of every two linear functions among the multiple linear functions includes a maximum of one value; and outputting a reference data point according to the target piecewise linear function, where the reference data point includes a point of continuity and a point of discontinuity of the target piecewise linear function. In this way, a maximum error, a target piecewise linear function is further determined according to the multiple line segments, and a point of continuity and a point of discontinuity of the target piecewise linear function are used to represent compressed flow data.

    SIMILARITY MEASUREMENT METHOD AND DEVICE
    8.
    发明申请

    公开(公告)号:US20170364478A1

    公开(公告)日:2017-12-21

    申请号:US15694559

    申请日:2017-09-01

    IPC分类号: G06F17/12 G06F17/30 G06F17/16

    摘要: A similarity measurement method includes: obtaining a directional relationship between nodes in a network, and determining a transition matrix; calculating a constraint matrix according to the transition matrix and an obtained attenuation factor; constructing a system of linear equations, where a coefficient matrix of the system of linear equations is the constraint matrix, and a variable of the system of linear equations is a correction vector; solving the system of linear equations by means of iteration by using a Jacobi method, and determining the correction vector; and calculating similarities between the nodes according to the transition matrix, the attenuation factor, and a diagonal correction matrix that is generated according to the correction vector. In the method, the correction vector is determined by using the Jacobi method, and further the similarities between the nodes may be calculated.

    Machine learning model training method and apparatus

    公开(公告)号:US12067483B2

    公开(公告)日:2024-08-20

    申请号:US16431393

    申请日:2019-06-04

    摘要: Embodiments of the present invention provide a machine learning model training method, including: obtaining target task training data and N categories of support task training data; inputting the target task training data and the N categories of support task training data into a memory model to obtain target task training feature data and N categories of support task training feature data; training the target task model based on the target task training feature data and obtaining a first loss of the target task model, and separately training respectively corresponding support task models based on the N categories of support task training feature data and obtaining respective second losses of the N support task models; and updating the memory model, the target task model, and the N support task models based on the first loss and the respective second losses of the N support task models.