公开(公告)号：US10387989B2

公开(公告)日：2019-08-20

申请号：US15628480

申请日：2017-06-20

Applicant: Google LLC

Inventor： Albert Meixner ,  Hyunchul Park ,  William R. Mark ,  Daniel Frederic Finchelstein ,  Ofer Shacham

IPC: G06F8/41 ,  G06F9/50 ,  G06T1/20

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for restructuring an image processing pipeline. The method includes compiling program code targeted for an image processor having programmable stencil processors composed of respective two-dimensional execution lane and shift register circuit structures. The program code is to implement a directed acyclic graph and is composed of multiple kernels that are to execute on respective ones of the stencil processors, wherein the compiling includes performing any of: horizontal fusion of kernels; vertical fusion of kernels; fission of one of the kernels into multiple kernels; spatial partitioning of a kernel into multiple spatially partitioned kernels; or splitting the directed acyclic graph into smaller graphs.

公开(公告)号：US10387988B2

公开(公告)日：2019-08-20

申请号：US15389113

申请日：2016-12-22

Applicant: Google LLC

Inventor： Albert Meixner ,  Hyunchul Park ,  William R. Mark ,  Daniel Frederic Finchelstein ,  Ofer Shacham

IPC: G06T1/20 ,  G06F9/50 ,  G06F8/41

Abstract: A method is described. The method includes compiling program code targeted for an image processor having programmable stencil processors composed of respective two-dimensional execution lane and shift register circuit structures. The program code is to implement a directed acyclic graph and is composed of multiple kernels that are to execute on respective ones of the stencil processors, wherein the compiling includes any of: recognizing there are a different number of kernels in the program code than stencil processors in the image processor; recognizing that at least one of the kernels is more computationally intensive than another one of the kernels; and, recognizing that the program code has resource requirements that exceed the image processor's memory capacity. The compiling further includes in response to any of the recognizing above performing any of: horizontal fusion of kernels; vertical fusion of kernels; fission of one of the kernels into multiple kernels; spatial partitioning of a kernel into multiple spatially partitioned kernels; splitting the directed acyclic graph into smaller graphs.

公开(公告)号：US11196953B2

公开(公告)日：2021-12-07

申请号：US16735050

申请日：2020-01-06

Applicant: Google LLC

Inventor： Albert Meixner ,  Daniel Frederic Finchelstein ,  David Patterson ,  William R. Mark ,  Jason Rupert Redgrave ,  Ofer Shacham

IPC: H04N5/225 ,  H04N5/374 ,  G06T1/20 ,  H04N5/341 ,  G06F5/01 ,  G06F12/02 ,  G06F17/16

Abstract: A method is described that includes, on an image processor having a two dimensional execution lane array and a two dimensional shift register array, repeatedly shifting first content of multiple rows or columns of the two dimensional shift register array and repeatedly executing at least one instruction between shifts that operates on the shifted first content and/or second content that is resident in respective locations of the two dimensional shift register array that the shifted first content has been shifted into.

公开(公告)号：US20200154072A1

公开(公告)日：2020-05-14

申请号：US16735050

申请日：2020-01-06

Applicant: Google LLC

Inventor： Albert Meixner ,  Daniel Frederic Finchelstein ,  David Patterson ,  William R. Mark ,  Jason Rupert Redgrave ,  Ofer Shacham

IPC: H04N5/374 ,  G06F17/16 ,  G06F12/02 ,  G06F5/01 ,  H04N5/341 ,  G06T1/20

Abstract: A method is described that includes, on an image processor having a two dimensional execution lane array and a two dimensional shift register array, repeatedly shifting first content of multiple rows or columns of the two dimensional shift register array and repeatedly executing at least one instruction between shifts that operates on the shifted first content and/or second content that is resident in respective locations of the two dimensional shift register array that the shifted first content has been shifted into.

公开(公告)号：US20200020069A1

公开(公告)日：2020-01-16

申请号：US16529633

申请日：2019-08-01

Applicant: Google LLC

Inventor： Albert Meixner ,  Hyunchul Park ,  William R. Mark ,  Daniel Frederic Finchelstein ,  Ofer Shacham

IPC: G06T1/20 ,  G06F8/41 ,  G06F9/50

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for restructuring an image processing pipeline. The method includes compiling program code targeted for an image processor having programmable stencil processors composed of respective two-dimensional execution lane and shift register circuit structures. The program code is to implement a directed acyclic graph and is composed of multiple kernels that are to execute on respective ones of the stencil processors, wherein the compiling includes performing any of: horizontal fusion of kernels; vertical fusion of kernels; fission of one of the kernels into multiple kernels; spatial partitioning of a kernel into multiple spatially partitioned kernels; or splitting the directed acyclic graph into smaller graphs.

公开(公告)号：US20180234653A1

公开(公告)日：2018-08-16

申请号：US15946095

申请日：2018-04-05

Applicant: Google LLC

Inventor： Albert Meixner ,  Daniel Frederic Finchelstein ,  David Patterson ,  William R. Mark ,  Jason Rupert Redgrave ,  Ofer Shacham

IPC: H04N5/374 ,  H04N5/341 ,  G06F5/01 ,  G06F17/16 ,  G06F12/02 ,  G06T1/20

CPC classification number: H04N5/3742 ,  G06F5/015 ,  G06F12/0207 ,  G06F17/16 ,  G06T1/20 ,  H04N5/341

Abstract: A method is described that includes, on an image processor having a two dimensional execution lane array and a two dimensional shift register array, repeatedly shifting first content of multiple rows or columns of the two dimensional shift register array and repeatedly executing at least one instruction between shifts that operates on the shifted first content and/or second content that is resident in respective locations of the two dimensional shift register array that the shifted first content has been shifted into.

公开(公告)号：US12020027B2

公开(公告)日：2024-06-25

申请号：US17028097

申请日：2020-09-22

Applicant: Google LLC

Inventor： Ofer Shacham ,  David Patterson ,  William R. Mark ,  Albert Meixner ,  Daniel Frederic Finchelstein ,  Jason Rupert Redgrave

IPC: G06N3/06 ,  G06F9/30 ,  G06F9/38 ,  G06N3/045 ,  G06N3/063 ,  G06T1/60 ,  G06T5/20

CPC classification number: G06F9/3001 ,  G06F9/30032 ,  G06F9/30036 ,  G06F9/3885 ,  G06F9/3887 ,  G06N3/045 ,  G06N3/063 ,  G06T1/60 ,  G06T5/20 ,  G06T2200/28 ,  G06T2207/20084

Abstract: A method is described that includes executing a convolutional neural network layer on an image processor having an array of execution lanes and a two-dimensional shift register. The two-dimensional shift register provides local respective register space for the execution lanes. The executing of the convolutional neural network includes loading a plane of image data of a three-dimensional block of image data into the two-dimensional shift register. The executing of the convolutional neural network also includes performing a two-dimensional convolution of the plane of image data with an array of coefficient values by sequentially: concurrently multiplying within the execution lanes respective pixel and coefficient values to produce an array of partial products; concurrently summing within the execution lanes the partial products with respective accumulations of partial products being kept within the two dimensional register for different stencils within the image data; and, effecting alignment of values for the two-dimensional convolution within the execution lanes by shifting content within the two-dimensional shift register array.

公开(公告)号：US20200160809A1

公开(公告)日：2020-05-21

申请号：US16685388

申请日：2019-11-15

Applicant: Google LLC

Inventor： Albert Meixner ,  Neeti Desai ,  Dilan Manatunga ,  Jason Rupert Redgrave ,  William R. Mark

IPC: G09G5/00 ,  G09G5/02 ,  G06T1/20 ,  G06F12/06 ,  G06T1/60 ,  H04N1/32

Abstract: An image processor is described. The image processor includes an I/O unit to read input image data from external memory for processing by the image processor and to write output image data from the image processor into the external memory. The I/O unit includes multiple logical channel units. Each logical channel unit is to form a logical channel between the external memory and a respective producing or consuming component within the image processor. Each logical channel unit is designed to utilize reformatting circuitry and addressing circuitry. The addressing circuitry is to control addressing schemes applied to the external memory and reformatting of image data between external memory and the respective producing or consuming component. The reformatting circuitry is to perform the reformatting.

公开(公告)号：US10334194B2

公开(公告)日：2019-06-25

申请号：US15946095

申请日：2018-04-05

Applicant: Google LLC

Inventor： Albert Meixner ,  Daniel Frederic Finchelstein ,  David Patterson ,  William R. Mark ,  Jason Rupert Redgrave ,  Ofer Shacham

IPC: H04N5/225 ,  H04N5/374 ,  G06T1/20 ,  H04N5/341 ,  G06F5/01 ,  G06F12/02 ,  G06F17/16

Abstract: A method is described that includes, on an image processor having a two dimensional execution lane array and a two dimensional shift register array, repeatedly shifting first content of multiple rows or columns of the two dimensional shift register array and repeatedly executing at least one instruction between shifts that operates on the shifted first content and/or second content that is resident in respective locations of the two dimensional shift register array that the shifted first content has been shifted into.