CLUSTER-CONNECTED NEURAL NETWORK
    4.
    发明公开

    公开(公告)号:EP4009243A1

    公开(公告)日:2022-06-08

    申请号:EP21205305.2

    申请日:2021-10-28

    申请人: Deepcube Ltd.

    发明人: DAVID, Eli RUBIN, Eri

    摘要: A device, system, and method is provided for training or prediction using a cluster-connected neural network. The cluster-connected neural network may be divided into a plurality of clusters of artificial neurons connected by weights or convolutional channels connected by convolutional filters. Within each cluster is a locally dense sub-network of intra-cluster weights or filters with a majority of pairs of neurons or channels connected by intra-cluster weights or filters that are co-activated together as an activation block during training or prediction. Outside each cluster is a globally sparse network of inter-cluster weights or filters with a minority of pairs of neurons or channels separated by a cluster border across different clusters connected by inter-cluster weights or filters. Training or predicting is performed using the cluster-connected neural network.

    PARTIAL-ACTIVATION OF NEURAL NETWORK BASED ON HEAT-MAP OF NEURAL NETWORK ACTIVITY

    公开(公告)号:EP3965015A1

    公开(公告)日:2022-03-09

    申请号:EP21178886.4

    申请日:2021-06-10

    申请人: Deepcube Ltd.

    发明人: DAVID, Eli RUBIN, Eri

    IPC分类号: G06N3/04 G06N3/08

    摘要: A method and system for training or prediction of a neural network. A current value may be stored for each of a plurality of synapses or filters in the neural network. A historical metric of activity may be independently determined for each individual or group of the synapses or filters during one or more past iterations. A plurality of partial activations of the neural network may be iteratively executed. Each partial-activation iteration may activate a subset of the plurality of synapses or filters in the neural network. Each individual or group of synapses or filters may be activated in a portion of a total number of iterations proportional to the historical metric of activity independently determined for that individual or group of synapses or filters. Training or prediction of the neural network may be performed based on the plurality of partial activations of the neural network.

    TRAINING A STUDENT NEURAL NETWORK TO MIMIC A MENTOR NEURAL NETWORK WITH INPUTS THAT MAXIMIZE STUDENT-TO-MENTOR DISAGREEMENT

    公开(公告)号:EP3937086A1

    公开(公告)日:2022-01-12

    申请号:EP21178887.2

    申请日:2021-06-10

    申请人: Deepcube Ltd.

    发明人: DAVID, Eli RUBIN, Eri

    IPC分类号: G06N3/04 G06N3/08

    摘要: A method and system is provided for training a new neural network to mimic a target neural network without access to the target neural network or its original training dataset. The target neural network and the new neural network may be probed with input data to generate corresponding target and new output data. Input data may be detected that generates a maximum or above threshold difference between the corresponding target and new output data. A divergent probe training dataset may be generated comprising the input data that generates the maximum or above threshold difference and the corresponding target output data. The new neural network may be trained using the divergent probe training dataset to generate the target output data. The new neural network may be iteratively trained using an updated divergent probe training dataset dynamically adjusted as the new neural network changes during training.