CONFIGURABLE HARDWARE TO IMPLEMENT A CONVOLUTIONAL NEURAL NETWORK

    公开(公告)号:EP4439272A2

    公开(公告)日:2024-10-02

    申请号:EP24194712.6

    申请日:2017-05-03

    IPC分类号: G06F7/544

    摘要: Configurable hardware for implementing a convolutional neural network (CNN) comprising: a memory interface configured to receive, from external memory, weights and input data to be used in calculations within the CNN, as well as command information to control operation of the configurable hardware; a coefficient buffer controller configured to receive the weights from the memory interface and pass the weights to a coefficient buffer; an input buffer controller configured to receive the input data from the memory interface and pass the input data to a plurality of input buffers; a command decoder configured to decode the command information; a plurality of convolution engines configured to perform one or more convolution operations on the input data in the plurality of input buffers using the weights in the coefficient buffer; a plurality of accumulators configured to receive resultant outputs of the plurality of convolution engines and add the resultant outputs of the plurality of convolution engines to values stored in an accumulation buffer; the accumulation buffer configured to store accumulated results from the plurality of accumulators; a shared buffer; an activation module configured to perform at least one of a number of different activation functions on data in the accumulation buffer and store the resultant values in the shared buffer; a normalize module configured to perform one of a number of different normalizing functions on data in the shared buffer and store the results in the shared buffer; and a pool module configured to perform a pooling operation on data in the shared buffer and store the results in the shared buffer.

    METHODS AND SYSTEMS FOR ONLINE SELECTION OF NUMBER FORMATS FOR NETWORK PARAMETERS OF A NEURAL NETWORK

    公开(公告)号:EP4345692A1

    公开(公告)日:2024-04-03

    申请号:EP23200824.3

    申请日:2023-09-29

    摘要: Methods and neural network accelerator for online selection of number formats for network parameters of a neural network. The neural network accelerator comprises: at least one network processing hardware unit configured to receive network parameters for layers of the neural network and perform one or more neural network operations on the received network parameters in accordance with the neural network; a statistics collection hardware unit configured to collect one or more statistics on a first set of network parameters for a layer while the neural network accelerator is performing a pass of the neural network; and a format conversion hardware unit configured to convert a second set of network parameters to a number format selected based on the collected one or more statistics, the second set of network parameters comprising (i) the first set of network parameters and/or another set of network parameters for the layer, or (ii) a set of network parameters for a subsequent pass of the neural network corresponding to the first set of network parameters.

    ADAPTIVE SHARPENING FOR BLOCKS OF PIXELS
    3.
    发明公开

    公开(公告)号:EP4345739A1

    公开(公告)日:2024-04-03

    申请号:EP23200311.1

    申请日:2023-09-28

    IPC分类号: G06T5/75

    摘要: There are provided methods and processing modules for applying adaptive sharpening, for a block of input pixels, to determine a block of output pixels. A block of sharp pixels is obtained based on the block of input pixels, the block of sharp pixels being for representing a sharp version of the block of output pixels. One or more indications of contrast for the block of input pixels is determined. Each of the output pixels of the block of output pixels is determined by performing a respective weighted sum of: (i) a corresponding input pixel in the block of input pixels and (ii) a corresponding sharp pixel in the block of sharp pixels. The weights of the weighted sums are based on the determined one or more indications of contrast for the block of input pixels.

    ADAPTIVE SHARPENING FOR BLOCKS OF UPSAMPLED PIXELS

    公开(公告)号:EP4345734A1

    公开(公告)日:2024-04-03

    申请号:EP23200310.3

    申请日:2023-09-28

    IPC分类号: G06T3/4053 G06T5/75

    摘要: Methods and processing modules are provided for applying adaptive sharpening, for a block of input pixels for which upsampling is performed, to determine a block of output pixels. A block of upsampled pixels is obtained based on the block of input pixels. One or more range kernels is determined based on a plurality of upsampled pixels of the block of upsampled pixels. Each of the one or more range kernels is combined with a sharpening kernel to determine one or more bilateral sharpening kernels. The one or more bilateral sharpening kernels are used to determine the output pixels of the block of output pixels.

    LEARNED IMAGE TRANSFORMATION METHODS AND SYSTEMS IN GRAPHICS RENDERING

    公开(公告)号:EP4446975A1

    公开(公告)日:2024-10-16

    申请号:EP24166007.5

    申请日:2024-03-25

    IPC分类号: G06T5/60 G06N3/088 G06T15/50

    摘要: The present disclosure pertains to learned image transformation methods and systems in graphics rendering. There is provided a method of transforming rendered frames in a graphics processing system to obtain enhanced frames with desired characteristics of a set of target images. The method comprises selecting a plurality of shaders, each defined by a parametrized mathematical function arranged to replicate a particular visual characteristic. For each shader: parameters of the parametrized mathematical function have been derived in dependence on a set of target images so that the shader is arranged to impose, when applied to a frame, its respective particular visual characteristic in dependence on an extent to which the particular visual characteristic is exhibited in the target images. The method further comprises combining the plurality of shaders to form a pipeline, obtaining one or more rendered frames, applying the pipeline to at least a portion of the one or more rendered frames to obtain enhanced frames, and outputting for display the enhanced frames, wherein the enhanced frames exhibit visual characteristics of the target images.

    UPSAMPLING BLOCKS OF PIXELS
    6.
    发明公开

    公开(公告)号:EP4350607A1

    公开(公告)日:2024-04-10

    申请号:EP23200314.5

    申请日:2023-09-28

    IPC分类号: G06T5/73 G06T3/4053

    摘要: Methods and processing modules are provided for upsampling a block of input pixels to determine a block of upsampled pixels. At least one of the upsampled pixels is a diagonal pixel, wherein a diagonal pixel is at a position that is not in any of the rows nor in any of the columns of input pixels in the block of input pixels. Indications of image gradients are determined for the block of input pixels (S404). The determined indications of image gradients are used to determine one or more weighting parameters which are indicative of weights of a diagonal kernel (S406). The upsampled pixels of the block of upsampled pixels are determined by applying kernels to the block of input pixels, wherein the diagonal pixel in the block of upsampled pixels is determined by applying the diagonal kernel to the block of input pixels in accordance with the determined one or more weighting parameters (S408).

    LEARNABLE IMAGE TRANSFORMATION TRAINING METHODS AND SYSTEMS IN GRAPHICS RENDERING

    公开(公告)号:EP4446996A1

    公开(公告)日:2024-10-16

    申请号:EP24166000.0

    申请日:2024-03-25

    摘要: The present disclosure pertains to learnable image transformation training methods and systems in graphics rendering. There is provided a method for training a frame transformation pipeline being part of a graphics processing system and configured to transform rendered frames to produce enhanced frames comprising visual characteristics exhibited in a set of target images. The frame transformation pipeline comprises one or more shaders, defined by a parametrized mathematical function capable of replicating a particular visual characteristic. The training method comprises: receiving input images and target images; applying each shader to the input images to obtain candidate frames, and calculating, at a parametrized discriminator, a similarity indication between characteristics of the candidate frames and the target images. The method further comprises, in dependence on the indication, a parameter update step to parameters of the discriminator and one or more parametrized mathematical functions, wherein the parameter update step is configured to derive parameters the parametrized mathematical function so that the one or more shaders is arranged to impose their respective particular visual characteristics in dependence on an extent to which visual characteristic is exhibited in the target images.

    VARIABLE INPUT SHAPES AT RUNTIME
    8.
    发明公开

    公开(公告)号:EP4404102A1

    公开(公告)日:2024-07-24

    申请号:EP23215380.9

    申请日:2023-12-08

    IPC分类号: G06N3/063 G06N3/0464

    摘要: A method of implementing in hardware a dynamic neural network for operation on an input tensor having a variable dimension, the method comprising: receiving a representation of the dynamic neural network; transforming the representation of the dynamic neural network into a static network adapted to operate on a fixed size input, the static network being adapted to perform operations on the fixed size input which are equivalent to the operations performed by the dynamic neural network on its input tensor; and implementing a plurality of instances of the static network in hardware for operation on an input tensor split into a sequence of overlapping fixed size inputs along its variable dimension, each instance of the static network being arranged to operate on a respective fixed size input of the sequence.

    ADAPTIVE SHARPENING FOR BLOCKS OF PIXELS
    9.
    发明公开

    公开(公告)号:EP4345740A1

    公开(公告)日:2024-04-03

    申请号:EP23200313.7

    申请日:2023-09-28

    IPC分类号: G06T5/75 G06T3/4053

    摘要: There are provided methods and processing modules for applying adaptive sharpening, for a block of input pixels (802) for which processing is performed, to determine a block of output pixels (804). A block of non-sharp processed pixels is obtained based on the block of input pixels (802), the block of non-sharp processed pixels being for representing a non-sharp version of the block of output pixels. A block of sharp processed pixels is obtained based on the block of input pixels (802), the block of sharp processed pixels being for representing a sharp version of the block of output pixels. One or more indications of contrast for the block of input pixels (802) is determined. Each of the output pixels of the block of output pixels (804) is determined by performing a respective weighted sum of: (i) a corresponding non-sharp processed pixel in the block of non-sharp processed pixels and (ii) a corresponding sharp processed pixel in the block of sharp processed pixels. The weights of the weighted sums are based on the determined one or more indications of contrast for the block of input pixels (802).