公开(公告)号：US12206845B2

公开(公告)日：2025-01-21

申请号：US17466176

申请日：2021-09-03

Applicant: NVIDIA Corporation

Inventor： Keshava Prasad ,  Hassane Samir Azar ,  Vinayak Pore

IPC: H04N19/117 ,  H04N19/164 ,  H04N21/2662

Abstract: In various examples, a deep neural network (DNN) based pre-filter for content streaming applications is used to dynamically adapt scene entropy (e.g., complexity) in response to changing network or system conditions of an end-user device. For example, where network and/or system performance issues or degradation are identified, the DNN may be implemented as a frame pre-filter to reduce the complexity or entropy of the frame prior to streaming—thereby allowing the frame to be streamed at a reduced bit rate without requiring a change in resolution. The DNN-based pre-filter may be tuned to maintain image detail along object, boundary, and/or surface edges such that scene navigation—such as by a user participating in an instance of an application—may be easier and more natural to the user.

公开(公告)号：US20250168333A1

公开(公告)日：2025-05-22

申请号：US19029177

申请日：2025-01-17

Applicant: NVIDIA Corporation

Inventor： Keshava Prasad ,  Hassane Samir Azar ,  Vinayak Pore

IPC: H04N19/117 ,  H04N19/164 ,  H04N21/2662

Abstract: In various examples, a deep neural network (DNN) based pre-filter for content streaming applications is used to dynamically adapt scene entropy (e.g., complexity) in response to changing network or system conditions of an end-user device. For example, where network and/or system performance issues or degradation are identified, the DNN may be implemented as a frame pre-filter to reduce the complexity or entropy of the frame prior to streaming-thereby allowing the frame to be streamed at a reduced bit rate without requiring a change in resolution. The DNN-based pre-filter may be tuned to maintain image detail along object, boundary, and/or surface edges such that scene navigation—such as by a user participating in an instance of an application—may be easier and more natural to the user.

公开(公告)号：US20230085156A1

公开(公告)日：2023-03-16

申请号：US17466176

申请日：2021-09-03

Applicant: NVIDIA Corporation

Inventor： Keshava Prasad ,  Hassane Samir Azar ,  Vinayak Pore

IPC: H04N19/117 ,  H04N19/164

Abstract: In various examples, a deep neural network (DNN) based pre-filter for content streaming applications is used to dynamically adapt scene entropy (e.g., complexity) in response to changing network or system conditions of an end-user device. For example, where network and/or system performance issues or degradation are identified, the DNN may be implemented as a frame pre-filter to reduce the complexity or entropy of the frame prior to streaming—thereby allowing the frame to be streamed at a reduced bit rate without requiring a change in resolution. The DNN-based pre-filter may be tuned to maintain image detail along object, boundary, and/or surface edges such that scene navigation—such as by a user participating in an instance of an application—may be easier and more natural to the user.

公开(公告)号：US11102516B2

公开(公告)日：2021-08-24

申请号：US15043978

申请日：2016-02-15

Applicant: Nvidia Corporation

Inventor： Thrinadh Kottana ,  Vinayak Pore ,  Chirayu Garg ,  Soumen Kumar Dey

IPC: H04N19/895

Abstract: A viewing device, a method of displaying streamed data frames and a client viewing device are disclosed herein. In one embodiment, the video viewing device includes: (1) a screen, (2) a decoder configured to decode a data frame received in a bitstream from a transmitter to provide a decoded data frame, and (3) an error concealer configured to either discard the decoded data frame or select the decoded data frame for display on the screen based on a complexity of the decoded data frame.

公开(公告)号：US12149708B2

公开(公告)日：2024-11-19

申请号：US17402953

申请日：2021-08-16

Applicant: NVIDIA Corporation

Inventor： Ravi Kumar Boddeti ,  Vinayak Pore ,  Hassane Samir Azar ,  Prashant Sohani

IPC: H04N19/00 ,  G06N20/00 ,  H04N19/127 ,  H04N19/142 ,  H04N19/166 ,  H04N19/65

Abstract: In various examples, machine learning of encoding parameter values for a network is performed using a video encoder. Feedback associated with streaming video encoded by a video encoder over a network may be applied to an MLM(s). Using such feedback, the MLM(s) may predict a value(s) of an encoding parameter(s). The video encoder may then use the value to encode subsequent video data for the streaming. By using the video encoder in training, the MLM(s) may learn based on actual encoded parameter values of the video encoder. The MLM(s) may be trained via reinforcement learning based on video encoded by the video encoder. A rewards metric(s) may be used to train the MLM(s) using data generated or applied to the physical network in which the MLM(s) is to be deployed and/or a simulation thereof. Penalty metric(s) (e.g., the quantity of dropped frames) may also be used to train the MLM(s).

公开(公告)号：US20230048189A1

公开(公告)日：2023-02-16

申请号：US17402953

申请日：2021-08-16

Applicant: NVIDIA Corporation

Inventor： Ravi kumar Boddeti ,  Vinayak Pore ,  Hassane Samir Azar ,  Prashant Sohani

IPC: H04N19/166 ,  G06N20/00 ,  H04N19/65 ,  H04N19/127 ,  H04N19/142

Abstract: In various examples, machine learning of encoding parameter values for a network is performed using a video encoder. Feedback associated with streaming video encoded by a video encoder over a network may be applied to an MLM(s). Using such feedback, the MLM(s) may predict a value(s) of an encoding parameter(s). The video encoder may then use the value to encode subsequent video data for the streaming. By using the video encoder in training, the MLM(s) may learn based on actual encoded parameter values of the video encoder. The MLM(s) may be trained via reinforcement learning based on video encoded by the video encoder. A rewards metric(s) may be used to train the MLM(s) using data generated or applied to the physical network in which the MLM(s) is to be deployed and/or a simulation thereof. Penalty metric(s) (e.g., the quantity of dropped frames) may also be used to train the MLM(s).

公开(公告)号：US11212539B2

公开(公告)日：2021-12-28

申请号：US16048120

申请日：2018-07-27

Applicant: NVIDIA Corporation

Inventor： Sarvesh Satavalekar ,  Gordon Grigor ,  Vinayak Pore ,  Gajanan Bhat ,  Mohan Nimaje ,  Soumen Dey ,  Sameer Gumaste

IPC: H04N19/182 ,  G06T7/11 ,  G06N3/02 ,  H04N19/33 ,  H04N19/186 ,  H04N1/64 ,  H04N9/67 ,  H04N1/413

Abstract: Systems and methods for efficient lossless compression of captured raw image information are presented. A method can comprise: receiving raw image data from an image capture device, segregating the pixel data into a base layer portion and an enhanced layer portion, reconfiguring the base layer portion expressed in the first color space values from a raw capture format into a pseudo second color space compression mechanism compatible format, and compressing the reconfigured base layer portion of first color space values. The raw image data can include pixel data are expressed in first color space values. The segregation can be based upon various factors, including a compression benefits analysis of a boundary location between the base layer portion and enhanced layer portion. The reconfiguring the base layer portion can include separating the base layer portion based upon multiple components within the raw data; and forming base layer video frames from the multiple components.

公开(公告)号：US20150208079A1

公开(公告)日：2015-07-23

申请号：US14160643

申请日：2014-01-22

Applicant: Nvidia Corporation

Inventor： Vinayak Pore ,  Shashank Garg ,  Sarvesh Satavalekar ,  Thomas J. Meier

IPC: H04N19/50

CPC classification number: H04N19/50 ,  H04N19/107 ,  H04N19/157 ,  H04N19/172 ,  H04N19/176

Abstract: An enhanced display encoder system for a video stream source includes an enhanced video encoder that has parallel intra frame and inter frame encoding units for encoding a video frame, wherein an initial number of macroblocks is encoded to determine a scene change status of the video frame. Additionally, a video frame history unit determines an intra frame update status for the video frame from a past number of video frames, and an encoder selection unit selects the intra frame or inter frame encoding unit for further encoding of the video frame to support a wireless transmission based on the scene change status and the intra frame update status. A method of enhanced video frame encoding for video stream sourcing is also provided.

Abstract translation: 用于视频流源的增强型显示编码器系统包括具有用于对视频帧进行编码的并行帧内和帧间编码单元的增强型视频编码器，其中编码初始数量的宏块以确定视频帧的场景改变状态。 此外，视频帧历史单元从过去数量的视频帧确定视频帧的帧内更新状态，并且编码器选择单元选择帧内或帧间编码单元以进一步编码视频帧以支持无线 基于场景变化状态和帧内更新状态的传输。 还提供了用于视频流源的增强视频帧编码的方法。

公开(公告)号：US20250068744A1

公开(公告)日：2025-02-27

申请号：US18949551

申请日：2024-11-15

Applicant: NVIDIA Corporation

Inventor： Ravi kumar Boddeti ,  Vinayak Pore ,  Hassane Samir Azar ,  Prashant Sohani

IPC: G06F21/57 ,  G06F21/53 ,  H04W12/37

Abstract: In various examples, machine learning of encoding parameter values for a network is performed using a video encoder. Feedback associated with streaming video encoded by a video encoder over a network may be applied to an MLM(s). Using such feedback, the MLM(s) may predict a value(s) of an encoding parameter(s). The video encoder may then use the value to encode subsequent video data for the streaming. By using the video encoder in training, the MLM(s) may learn based on actual encoded parameter values of the video encoder. The MLM(s) may be trained via reinforcement learning based on video encoded by the video encoder. A rewards metric(s) may be used to train the MLM(s) using data generated or applied to the physical network in which the MLM(s) is to be deployed and/or a simulation thereof. Penalty metric(s) (e.g., the quantity of dropped frames) may also be used to train the MLM(s).

公开(公告)号：US11889122B2

公开(公告)日：2024-01-30

申请号：US17409414

申请日：2021-08-23

Applicant: Nvidia Corporation

Inventor： Thrinadh Kottana ,  Vinayak Pore ,  Chirayu Garg ,  Soumen Kumar Dey

IPC: H04N19/895

CPC classification number: H04N19/895

Abstract: A technique for streaming and a client device that uses the technique are disclosed herein. The disclosed technique determines context complexity of streamed data and determines whether to discard or select the streamed data for a future reference frame based on the context complexity of the streamed data. The streamed data is discarded if the content complexity is higher than a content complexity threshold, and the streamed data is selected if the content complexity is not higher than a content complexity threshold. This is based on the realization that error propagation in the case of a less complex video sequence is not very bothersome to the end user experience whereas corruption will be very severe in cases of highly complex sequences.