公开(公告)号：US10733956B2

公开(公告)日：2020-08-04

申请号：US16685388

申请日：2019-11-15

Applicant: Google LLC

Inventor： Albert Meixner ,  Neeti Desai ,  Dilan Manatunga ,  Jason Rupert Redgrave ,  William R. Mark

IPC: H04N1/32 ,  G06T1/60 ,  G06F12/06 ,  G09G5/00 ,  G09G5/02 ,  G06T1/20

Abstract: An image processor is described. The image processor includes an I/O unit to read input image data from external memory for processing by the image processor and to write output image data from the image processor into the external memory. The I/O unit includes multiple logical channel units. Each logical channel unit is to form a logical channel between the external memory and a respective producing or consuming component within the image processor. Each logical channel unit is designed to utilize reformatting circuitry and addressing circuitry. The addressing circuitry is to control addressing schemes applied to the external memory and reformatting of image data between external memory and the respective producing or consuming component. The reformatting circuitry is to perform the reformatting.

公开(公告)号：US10719905B2

公开(公告)日：2020-07-21

申请号：US16547801

申请日：2019-08-22

Applicant: Google LLC

Inventor： Qiuling Zhu ,  Ofer Shacham ,  Albert Meixner ,  Jason Rupert Redgrave ,  Daniel Frederic Finchelstein ,  David Patterson ,  Neeti Desai ,  Donald Stark ,  Edward Chang ,  William R. Mark

IPC: G06T1/20 ,  G06T1/60 ,  H04N5/378 ,  H04N5/91

Abstract: An apparatus is described. The apparatus includes an image processing unit. The image processing unit includes a plurality of stencil processor circuits each comprising an array of execution unit lanes coupled to a two-dimensional shift register array structure to simultaneously process multiple overlapping stencils through execution of program code. The image processing unit includes a plurality of sheet generators respectively coupled between the plurality of stencil processors and the network. The sheet generators are to parse input line groups of image data into input sheets of image data for processing by the stencil processors, and, to form output line groups of image data from output sheets of image data received from the stencil processors. The image processing unit includes a plurality of line buffer units coupled to the network to pass line groups in a direction from producing stencil processors to consuming stencil processors to implement an overall program flow.

公开(公告)号：US20200159494A1

公开(公告)日：2020-05-21

申请号：US16687488

申请日：2019-11-18

Applicant: Google LLC

Inventor： Artem Vasilyev ,  Albert Meixner ,  Jason Rupert Redgrave

IPC: G06F7/50 ,  G06T1/20 ,  G11C19/38 ,  G06F9/38 ,  G06F9/30

Abstract: An execution unit is described. The execution unit includes an arithmetic logic unit (ALU) circuit having a first input to receive a first value and a second input to receive a second value. The ALU circuit includes circuitry to determine an absolute value of the first value and to add the absolute value to the second value. The first input is coupled to a first data path having register space and an output of another ALU of the execution unit circuit as alternative sources of the first value. The second input is coupled to a second data path having the register space as a source for the second value.

公开(公告)号：US10417732B2

公开(公告)日：2019-09-17

申请号：US15599348

申请日：2017-05-18

Applicant: Google LLC

Inventor： Qiuling Zhu ,  Ofer Shacham ,  Albert Meixner ,  Jason Rupert Redgrave ,  Daniel Frederic Finchelstein ,  David Patterson ,  Neeti Desai ,  Donald Stark ,  Edward Chang ,  William Mark

IPC: G06T1/20 ,  G06T1/60 ,  H04N5/378 ,  H04N5/91

Abstract: An apparatus is described. The apparatus includes an image processing unit. The image processing unit includes a plurality of stencil processor circuits each comprising an array of execution unit lanes coupled to a two-dimensional shift register array structure to simultaneously process multiple overlapping stencils through execution of program code. The image processing unit includes a plurality of sheet generators respectively coupled between the plurality of stencil processors and the network. The sheet generators are to parse input line groups of image data into input sheets of image data for processing by the stencil processors, and, to form output line groups of image data from output sheets of image data received from the stencil processors. The image processing unit includes a plurality of line buffer units coupled to the network to pass line groups in a direction from producing stencil processors to consuming stencil processors to implement an overall program flow.

公开(公告)号：US10387989B2

公开(公告)日：2019-08-20

申请号：US15628480

申请日：2017-06-20

Applicant: Google LLC

Inventor： Albert Meixner ,  Hyunchul Park ,  William R. Mark ,  Daniel Frederic Finchelstein ,  Ofer Shacham

IPC: G06F8/41 ,  G06F9/50 ,  G06T1/20

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for restructuring an image processing pipeline. The method includes compiling program code targeted for an image processor having programmable stencil processors composed of respective two-dimensional execution lane and shift register circuit structures. The program code is to implement a directed acyclic graph and is composed of multiple kernels that are to execute on respective ones of the stencil processors, wherein the compiling includes performing any of: horizontal fusion of kernels; vertical fusion of kernels; fission of one of the kernels into multiple kernels; spatial partitioning of a kernel into multiple spatially partitioned kernels; or splitting the directed acyclic graph into smaller graphs.

公开(公告)号：US10387988B2

公开(公告)日：2019-08-20

申请号：US15389113

申请日：2016-12-22

Applicant: Google LLC

Inventor： Albert Meixner ,  Hyunchul Park ,  William R. Mark ,  Daniel Frederic Finchelstein ,  Ofer Shacham

IPC: G06T1/20 ,  G06F9/50 ,  G06F8/41

Abstract: A method is described. The method includes compiling program code targeted for an image processor having programmable stencil processors composed of respective two-dimensional execution lane and shift register circuit structures. The program code is to implement a directed acyclic graph and is composed of multiple kernels that are to execute on respective ones of the stencil processors, wherein the compiling includes any of: recognizing there are a different number of kernels in the program code than stencil processors in the image processor; recognizing that at least one of the kernels is more computationally intensive than another one of the kernels; and, recognizing that the program code has resource requirements that exceed the image processor's memory capacity. The compiling further includes in response to any of the recognizing above performing any of: horizontal fusion of kernels; vertical fusion of kernels; fission of one of the kernels into multiple kernels; spatial partitioning of a kernel into multiple spatially partitioned kernels; splitting the directed acyclic graph into smaller graphs.

公开(公告)号：US20190188824A1

公开(公告)日：2019-06-20

申请号：US16272819

申请日：2019-02-11

Applicant: Google LLC

Inventor： Albert Meixner ,  Hyunchul Park ,  Qiuling Zhu ,  Jason Rupert Redgrave

IPC: G06T1/60 ,  G06F9/38 ,  G06T1/20

Abstract: A method is described. The method includes repeatedly loading a next sheet of image data from a first location of a memory into a two dimensional shift register array. The memory is locally coupled to the two-dimensional shift register array and an execution lane array having a smaller dimension than the two-dimensional shift register array along at least one array axis. The loaded next sheet of image data keeps within an image area of the two-dimensional shift register array. The method also includes repeatedly determining output values for the next sheet of image data through execution of program code instructions along respective lanes of the execution lane array, wherein, a stencil size used in determining the output values encompasses only pixels that reside within the two-dimensional shift register array.

公开(公告)号：US10304156B2

公开(公告)日：2019-05-28

申请号：US15625972

申请日：2017-06-16

Applicant: Google LLC

Inventor： Albert Meixner ,  Hyunchul Park ,  Qiuling Zhu ,  Jason Rupert Redgrave

IPC: G09G3/36 ,  G06T1/60 ,  G06F9/38 ,  G06T1/20

Abstract: A method is described. The method includes repeatedly loading a next sheet of image data from a first location of a memory into a two dimensional shift register array. The memory is locally coupled to the two-dimensional shift register array and an execution lane array having a smaller dimension than the two-dimensional shift register array along at least one array axis. The loaded next sheet of image data keeps within an image area of the two-dimensional shift register array. The method also includes repeatedly determining output values for the next sheet of image data through execution of program code instructions along respective lanes of the execution lane array, wherein, a stencil size used in determining the output values encompasses only pixels that reside within the two-dimensional shift register array.

公开(公告)号：US09978116B2

公开(公告)日：2018-05-22

申请号：US15598082

申请日：2017-05-17

Applicant: Google LLC

Inventor： Albert Meixner ,  Daniel Frederic Finchelstein ,  David Patterson ,  William Mark ,  Jason Rupert Redgrave ,  Ofer Shacham

IPC: G06K9/00 ,  G06T1/20 ,  G11C19/28

CPC classification number: G06T1/20 ,  G06K9/00986 ,  G11C19/28

Abstract: A method is described that includes, on an image processor having a two dimensional execution lane array and a two dimensional shift register array, doubling a simultaneous shift amount of multiple rows or columns of the two dimensional shift register array with each next iteration. The method also includes executing one or more instructions within respective lanes of the two dimensional execution lane array in between shifts of iterations. Another method is described that includes, on an image processor having a two dimensional execution lane array and a two dimensional shift register array, repeatedly executing one or more instructions within respective lanes of the execution lane array that select between content in different registers of a same array location in between repeated simultaneous shifts of multiple rows or columns of data in the two dimensional shift register array.

公开(公告)号：US12020027B2

公开(公告)日：2024-06-25

申请号：US17028097

申请日：2020-09-22

Applicant: Google LLC

Inventor： Ofer Shacham ,  David Patterson ,  William R. Mark ,  Albert Meixner ,  Daniel Frederic Finchelstein ,  Jason Rupert Redgrave

IPC: G06N3/06 ,  G06F9/30 ,  G06F9/38 ,  G06N3/045 ,  G06N3/063 ,  G06T1/60 ,  G06T5/20

CPC classification number: G06F9/3001 ,  G06F9/30032 ,  G06F9/30036 ,  G06F9/3885 ,  G06F9/3887 ,  G06N3/045 ,  G06N3/063 ,  G06T1/60 ,  G06T5/20 ,  G06T2200/28 ,  G06T2207/20084

Abstract: A method is described that includes executing a convolutional neural network layer on an image processor having an array of execution lanes and a two-dimensional shift register. The two-dimensional shift register provides local respective register space for the execution lanes. The executing of the convolutional neural network includes loading a plane of image data of a three-dimensional block of image data into the two-dimensional shift register. The executing of the convolutional neural network also includes performing a two-dimensional convolution of the plane of image data with an array of coefficient values by sequentially: concurrently multiplying within the execution lanes respective pixel and coefficient values to produce an array of partial products; concurrently summing within the execution lanes the partial products with respective accumulations of partial products being kept within the two dimensional register for different stencils within the image data; and, effecting alignment of values for the two-dimensional convolution within the execution lanes by shifting content within the two-dimensional shift register array.