公开(公告)号：US11880261B2

公开(公告)日：2024-01-23

申请号：US17709720

申请日：2022-03-31

Applicant: Nvidia Corporation

Inventor： Evgeny Bolotin ,  Yaosheng Fu ,  Zi Yan ,  Gal Dalal ,  Shie Mannor ,  David Nellans

IPC: G06F1/324 ,  G06F11/34 ,  G06F1/20

CPC classification number: G06F1/324 ,  G06F1/206 ,  G06F11/3495

Abstract: A system, method, and apparatus of power management for computing systems are included herein that optimize individual frequencies of components of the computing systems using machine learning. The computing systems can be tightly integrated systems that consider an overall operating budget that is shared between the components of the computing system while adjusting the frequencies of the individual components. An example of an automated method of power management includes: (1) learning, using a power management (PM) agent, frequency settings for different components of a computing system during execution of a repetitive application, and (2) adjusting the frequency settings of the different components using the PM agent, wherein the adjusting is based on the repetitive application and one or more limitations corresponding to a shared operating budget for the computing system.

公开(公告)号：US20240010232A1

公开(公告)日：2024-01-11

申请号：US18318233

申请日：2023-05-16

Applicant: NVIDIA Corporation

Inventor： Peter Karkus ,  Boris Ivanovic ,  Shie Mannor ,  Marco Pavone

IPC: B60W60/00 ,  B60W30/095 ,  B60W50/00 ,  G06N3/08

CPC classification number: B60W60/0011 ,  B60W30/0956 ,  B60W50/0097 ,  G06N3/08 ,  B60W2554/4041 ,  B60W2554/4045

Abstract: In various examples, a motion planner include an analytical function to predict motion plans for a machine based on predicted trajectories of actors in an environment, where the predictions are differentiable with respect to parameters of a neural network of a motion predictor used to predict the trajectories. The analytical function may be used to determine candidate trajectories for the machine based on a predicted trajectory, to compute cost values for the candidate trajectories, and to select a reference trajectory from the candidate trajectories. For differentiability, a term of the analytical function may correspond to the predicted trajectory. A motion controller may use the reference trajectory to predict a control sequence for the machine using an analytical function trained to generate predictions that are differentiable with respect to at least one parameter of the analytical function used to compute the cost values.

公开(公告)号：US20230137205A1

公开(公告)日：2023-05-04

申请号：US17514735

申请日：2021-10-29

Applicant: Nvidia Corporation

Inventor： Yaosheng Fu ,  Shie Mannor ,  Evgeny Bolotin ,  David Nellans ,  Gal Dalal

IPC: G06F12/123 ,  G06N20/00 ,  G06T1/60

Abstract: Introduced herein is a technique that uses ML to autonomously find a cache management policy that achieves an optimal execution of a given workload of an application. Leveraging ML such as reinforcement learning, the technique trains an agent in an ML environment over multiple episodes of a stabilization process. For each time step in these training episodes, the agent executes the application while making an incremental change to the current policy, i.e., cache-residency statuses of memory address space associated with the workload, until the application can be executed at a stable level. The stable level of execution, for example, can be indicated by performance variations, such as standard deviations, between a certain number of neighboring measurement periods remaining within a certain threshold. The agent, who has been trained in the training episodes, infers the final cache management policy during the final, inferring episode.

公开(公告)号：US20250053826A1

公开(公告)日：2025-02-13

申请号：US18754007

申请日：2024-06-25

Applicant: NVIDIA Corporation

Inventor： Eli Alexander Meirom ,  Piotr Sielski ,  Gal Chechik ,  Alexandre Fender ,  Shie Mannor

IPC: G06N3/126 ,  G06N3/092

Abstract: A technique for solving combinatorial problems, such as vehicle routing for multiple vehicles integrates evolutionary algorithms and reinforcement learning. A genetic algorithm maintains a set of solutions for the problem and improves the solutions using mutation (modify a solution) and crossover (combine two solutions). The best solution is selected from the improved set of solutions. A system that integrates evolutionary algorithms, such as a genetic algorithm, and reinforcement learning comprises two components. A first component is a beam search technique for generating solutions using a reinforcement learning model. A second component augments a genetic algorithm using learning-based solutions that are generated by the reinforcement learning model. The learning-based solutions improve the diversity of the set which, in turn, improves the quality of the solutions computed by the genetic algorithm.

公开(公告)号：US20230041242A1

公开(公告)日：2023-02-09

申请号：US17959042

申请日：2022-10-03

Applicant: NVIDIA Corporation

Inventor： Shie Mannor ,  Chen Tessler ,  Yuval Shpigelman ,  Amit Mandelbaum ,  Gal Dalal ,  Doron Kazakov ,  Benjamin Fuhrer

IPC: H04L43/0817 ,  H04L43/067 ,  H04L43/0852 ,  G06N3/08 ,  H04L47/122 ,  G06K9/62 ,  H04L43/0882

Abstract: A reinforcement learning agent learns a congestion control policy using a deep neural network and a distributed training component. The training component enables the agent to interact with a vast set of environments in parallel. These environments simulate real world benchmarks and real hardware. During a learning process, the agent learns how maximize an objective function. A simulator may enable parallel interaction with various scenarios. As the trained agent encounters a diverse set of problems it is more likely to generalize well to new and unseen environments. In addition, an operating point can be selected during training which may enable configuration of the required behavior of the agent.

公开(公告)号：US20250053284A1

公开(公告)日：2025-02-13

申请号：US18232016

申请日：2023-08-09

Applicant: NVIDIA Corporation

Inventor： Shie Mannor ,  Gal Chechik

IPC: G06F3/04845 ,  G06F3/04815

Abstract: Apparatuses, systems, and techniques to identify one or more modifications to objects within an environment. In at least one embodiment, objects are identified in an image, based on extracted feedback information, using one or more machine learning models, for example, using direct and/or implicit feedback of user interaction with one or more objects in an environment.

公开(公告)号：US20220231933A1

公开(公告)日：2022-07-21

申请号：US17341210

申请日：2021-06-07

Applicant: NVIDIA Corporation

Inventor： Shie Mannor ,  Chen Tessler ,  Yuval Shpigelman ,  Amit Mandelbaum ,  Gal Dalal ,  Doron Kazakov ,  Benjamin Fuhrer

IPC: H04L12/26 ,  H04L12/803 ,  G06K9/62 ,  G06N3/08

Abstract: A reinforcement learning agent learns a congestion control policy using a deep neural network and a distributed training component. The training component enables the agent to interact with a vast set of environments in parallel. These environments simulate real world benchmarks and real hardware. During a learning process, the agent learns how maximize an objective function. A simulator may enable parallel interaction with various scenarios. As the trained agent encounters a diverse set of problems it is more likely to generalize well to new and unseen environments. In addition, an operating point can be selected during training which may enable configuration of the required behavior of the agent.

公开(公告)号：US20240406058A1

公开(公告)日：2024-12-05

申请号：US18629132

申请日：2024-04-08

Applicant: Nvidia Corporation

Inventor： Elad Alon ,  Eitan Zahavi ,  Gaby Diengott ,  Shie Mannor ,  Vadim Gechman

IPC: H04L41/0659 ,  H04L41/147 ,  H04L43/06 ,  H04L43/0811

Abstract: A network monitor may execute, or communicate with, one or more stored machine learning models that are trained to predict a failure probability for one or more ports and/or links within a network fabric. Systems and methods may monitor a set of ports and/or links to generate predictions for failure probabilities using a first trained model and low frequency telemetry data. For a subset of ports and/or links with failure probabilities exceeding a first threshold, high speed telemetry data may be used by a second trained model to generate predictions for failure probabilities for the subset of ports. Suspicious ports may then be isolated and undergo various remediation and/or monitoring actions prior to de-isolating the isolated ports.

公开(公告)号：US20240249458A1

公开(公告)日：2024-07-25

申请号：US18364982

申请日：2023-08-03

Applicant: NVIDIA Corporation

Inventor： Chen Tessler ,  Gal Chechik ,  Yoni Kasten ,  Shie Mannor ,  Jason Peng

IPC: G06T13/40 ,  G06N3/08 ,  G06T13/80

CPC classification number: G06T13/40 ,  G06N3/08 ,  G06T13/80

Abstract: A conditional adversarial latent model (CALM) process can be used to generate reference motions from a set of original reference movements to create a library of new movements for an agent. The agent can be a virtual representation various types of characters, animals, or objects. The CALM process can receive a set of reference movements and a requested movement. An encoder can be used to map the requested movement onto a latent space. A low-level policy can be employed to produce a series of latent space joint movements for the agent. A conditional discriminator can be used to provide feedback to the low-level policy to produce stationary distributions over the states of the agent. A high-level policy can be employed to provide a macro movement control over the low-level policy movements, such as providing direction in the environment. The high-level policy can utilize a reward or a finite-state machine function.

公开(公告)号：US20230237342A1

公开(公告)日：2023-07-27

申请号：US18158920

申请日：2023-01-24

Applicant: NVIDIA Corporation

Inventor： Shie Mannor ,  Gal Chechik ,  Gal Dalal ,  Assaf Joseph Hallak ,  Aviv Rosenberg

IPC: G06N3/092

CPC classification number: G06N3/092

Abstract: A method is performed by an agent operating in an environment. The method comprises computing a first value associated with each state of a number of states in the environment, determining a lookahead horizon for each state of the number of states in the environment based on the computed first value for each state of the number of states, applying a first policy to compute a second value associated with each state of at least one state in the number of states in the environment for the at least one state in the number of states based on the determined lookahead horizons for the number of states, and determining a second policy based on the first policy and the second value for each state of the number of states in the environment.