公开(公告)号：US11580736B2

公开(公告)日：2023-02-14

申请号：US16954068

申请日：2019-01-07

Applicant: DeepMind Technologies Limited

Inventor： Simon Osindero ,  Joao Carreira ,  Viorica Patraucean ,  Andrew Zisserman

IPC: G06V20/40 ,  G06N3/04 ,  G06T1/20 ,  G06N3/049

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for parallel processing of video frames using neural networks. One of the methods includes receiving a video sequence comprising a respective video frame at each of a plurality of time steps; and processing the video sequence using a video processing neural network to generate a video processing output for the video sequence, wherein the video processing neural network includes a sequence of network components, wherein the network components comprise a plurality of layer blocks each comprising one or more neural network layers, wherein each component is active for a respective subset of the plurality of time steps, and wherein each layer block is configured to, at each time step at which the layer block is active, receive an input generated at a previous time step and to process the input to generate a block output.

公开(公告)号：US11361546B2

公开(公告)日：2022-06-14

申请号：US17004814

申请日：2020-08-27

Applicant: DeepMind Technologies Limited

Inventor： Joao Carreira ,  Andrew Zisserman

IPC: G06K9/62 ,  G06N3/04 ,  G06N3/08 ,  G06T7/269 ,  G06V10/44 ,  G06V10/82 ,  G06V20/40 ,  G06V40/20

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing video data. An example system receives video data and generates optical flow data. An image sequence from the video data is provided to a first 3D spatio-temporal convolutional neural network to process the image data in at least three space-time dimensions and to provide a first convolutional neural network output. A corresponding sequence of optical flow image frames is provided to a second 3D spatio-temporal convolutional neural network to process the optical flow data in at least three space-time dimensions and to provide a second convolutional neural network output. The first and second convolutional neural network outputs are combined to provide a system output.

公开(公告)号：US20210034909A1

公开(公告)日：2021-02-04

申请号：US16995307

申请日：2020-08-17

Applicant: DeepMind Technologies Limited

Inventor： Maxwell Elliot Jaderberg ,  Karen Simonyan ,  Andrew Zisserman ,  Koray Kavukcuoglu

IPC: G06K9/52 ,  G06N3/04 ,  G06K9/46 ,  G06N3/08 ,  G06K9/03

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing inputs using an image processing neural network system that includes a spatial transformer module. One of the methods includes receiving an input feature map derived from the one or more input images, and applying a spatial transformation to the input feature map to generate a transformed feature map, comprising: processing the input feature map to generate spatial transformation parameters for the spatial transformation, and sampling from the input feature map in accordance with the spatial transformation parameters to generate the transformed feature map.

公开(公告)号：US20210027064A1

公开(公告)日：2021-01-28

申请号：US16954068

申请日：2019-01-07

Applicant: DeepMind Technologies Limited

Inventor： Simon Osindero ,  Joao Carreira ,  Viorica Patraucean ,  Andrew Zisserman

IPC: G06K9/00 ,  G06N3/04 ,  G06T1/20

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for parallel processing of video frames using neural networks. One of the methods includes receiving a video sequence comprising a respective video frame at each of a plurality of time steps; and processing the video sequence using a video processing neural network to generate a video processing output for the video sequence, wherein the video processing neural network includes a sequence of network components, wherein the network components comprise a plurality of layer blocks each comprising one or more neural network layers, wherein each component is active for a respective subset of the plurality of time steps, and wherein each layer block is configured to, at each time step at which the layer block is active, receive an input generated at a previous time step and to process the input to generate a block output.

公开(公告)号：US20180330185A1

公开(公告)日：2018-11-15

申请号：US16041567

申请日：2018-07-20

Applicant: DeepMind Technologies Limited

Inventor： Maxwell Elliot Jaderberg ,  Karen Simonyan ,  Andrew Zisserman ,  Koray Kavukcuoglu

IPC: G06K9/52 ,  G06N3/08 ,  G06K9/46 ,  G06K9/03 ,  G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing inputs using an image processing neural network system that includes a spatial transformer module. One of the methods includes receiving an input feature map derived from the one or more input images, and applying a spatial transformation to the input feature map to generate a transformed feature map, comprising: processing the input feature map to generate spatial transformation parameters for the spatial transformation, and sampling from the input feature map in accordance with the spatial transformation parameters to generate the transformed feature map.