MULTI-THREADING IN A VIDEO HARDWARE ENGINE

    公开(公告)号:US20220094948A1

    公开(公告)日:2022-03-24

    申请号:US17543183

    申请日:2021-12-06

    Abstract: A video hardware engine with multi-threading functionality is disclosed. The video hardware engine includes a video hardware accelerator unit and a controller. The controller is coupled to the video hardware accelerator unit. The controller operates in an encode mode and a decode mode. In the encode mode, the controller receives a plurality of frames and encode attributes associated with each frame of the plurality of frames. The encode attributes associated with a frame of the plurality of frames is processed to generate encode parameters associated with the frame. The video hardware accelerator unit is configured to process the frame based on the encode parameters to generate an output. The output of the video hardware accelerator unit is processed to generate a compressed bit-stream and an encode status. In decode mode, the controller receives a compressed bit-stream and decode attributes and generates a plurality of frames and a decode status.

    Reconfigurable image processing hardware pipeline

    公开(公告)号:US11276134B2

    公开(公告)日:2022-03-15

    申请号:US16847864

    申请日:2020-04-14

    Abstract: A reconfigurable image processing pipeline includes an image signal processor (ISP), a control processor, and a local memory. ISP processes raw pixel data for a frame based on an image processing parameter and provides lines of processed pixel data to control processor via a first interface. For each region of interest (ROI) in the frame, ISP generates auto-exposure and auto-white balance (2A) statistics based on the lines for the ROI and writes them to the local memory via a second interface. Control processor reads 2A statistics from the local memory, determines the image processing parameter based on them, and provides the image processing parameter to ISP. ISP also generates an integer N bin histogram for control processor, which sums a portion of the N total bins and compares the summed bin count to a lighting transition threshold. The image processing parameter is further based on the comparison.

    METHOD AND APPARATUS FOR REAL-TIME SAO PARAMETER ESTIMATION

    公开(公告)号:US20210211680A1

    公开(公告)日:2021-07-08

    申请号:US17212321

    申请日:2021-03-25

    Abstract: The disclosure provides a sample adaptive offset (SAO) encoder. The SAO encoder includes a statistics collection (SC) block and a rate distortion optimization (RDO) block coupled to the SC block. The SC block receives a set of deblocked pixels and a set of original pixels. The SC block categorizes each deblocked pixel of the set of deblocked pixels in at least one of a plurality of band and edge categories. The SC block estimates an error in each category as difference between a deblocked pixel of the set of deblocked pixels and corresponding original pixel of the set of original pixels. The RDO block determines a set of candidate offsets associated with each category and selects a candidate offset with a minimum RD cost. The minimum RD cost is used by a SAO type block and a decision block to generate final offsets for the SAO encoder.

    DELAYED DUPLICATE I-PICTURE FOR VIDEO CODING

    公开(公告)号:US20210058645A1

    公开(公告)日:2021-02-25

    申请号:US17093695

    申请日:2020-11-10

    Abstract: A method is provided that includes receiving pictures of a video sequence in a video encoder, and encoding the pictures to generate a compressed video bit stream that is transmitted to a video decoder in real-time, wherein encoding the pictures includes selecting a picture to be encoded as a delayed duplicate intra-predicted picture (DDI), wherein the picture would otherwise be encoded as an inter-predicted picture (P-picture), encoding the picture as an intra-predicted picture (I-picture) to generate the DDI, wherein the I-picture is reconstructed and stored for use as a reference picture for a decoder refresh picture, transmitting the DDI to the video decoder in non-real time, selecting a subsequent picture to be encoded as the decoder refresh picture, and encoding the subsequent picture in the compressed bit stream as the decoder refresh picture, wherein the subsequent P-picture is encoded as a P-picture predicted using the reference picture.

    Scheduling of external block based data processing tasks on a hardware thread scheduler

    公开(公告)号:US10908946B2

    公开(公告)日:2021-02-02

    申请号:US15396172

    申请日:2016-12-30

    Abstract: A data processing device is provided that includes a plurality of hardware data processing nodes, wherein each hardware data processing node performs a task, and a hardware thread scheduler including a plurality of hardware task schedulers configured to control execution of a respective task on a respective hardware data processing node of the plurality of hardware data processing nodes, and a proxy hardware task scheduler coupled to a data processing node external to the data processing device, wherein the proxy hardware task scheduler is configured to control execution of a task by the external data processing device, wherein the hardware thread scheduler is configurable to execute a thread of tasks, the tasks including the task controlled by the proxy hardware task scheduler and a first task controlled by a first hardware task scheduler of the plurality of hardware task schedulers.

    Hierarchical data organization for dense optical flow processing in a computer vision system

    公开(公告)号:US10824877B2

    公开(公告)日:2020-11-03

    申请号:US15638142

    申请日:2017-06-29

    Abstract: A computer vision system is provided that includes an image generation device configured to capture consecutive two dimensional (2D) images of a scene, a first memory configured to store the consecutive 2D images, a second memory configured to store a growing window of consecutive rows of a reference image and a growing window of consecutive rows of a current image, wherein the reference image and the current image are a pair of consecutive 2D images stored in the first memory, a third memory configured to store a sliding window of pixels fetched from the growing window of the reference image, wherein the pixels in the sliding window are stored in tiles, and a dense optical flow engine (DOFE) configured to determine a dense optical flow map for the pair of consecutive 2D images, wherein the DOFE uses the sliding window as a search window for pixel correspondence searches.

    METHODS AND SYSTEMS FOR ANALYZING IMAGES IN CONVOLUTIONAL NEURAL NETWORKS

    公开(公告)号:US20200302217A1

    公开(公告)日:2020-09-24

    申请号:US16898972

    申请日:2020-06-11

    Abstract: A method for analyzing images to generate a plurality of output features includes receiving input features of the image and performing Fourier transforms on each input feature. Kernels having coefficients of a plurality of trained features are received and on-the-fly Fourier transforms (OTF-FTs) are performed on the coefficients in the kernels. The output of each Fourier transform and each OTF-FT are multiplied together to generate a plurality of products and each of the products are added to produce one sum for each output feature. Two-dimensional inverse Fourier transforms are performed on each sum.

Patent Agency Ranking