公开(公告)号：US20220374723A1

公开(公告)日：2022-11-24

申请号：US17316564

申请日：2021-05-10

Applicant: NVIDIA Corporation

Inventor： Valts Blukis ,  Christopher Jason Paxton ,  Animesh Garg ,  Dieter Fox

IPC: G06N5/00 ,  G06F16/332 ,  G06K9/62 ,  G06N3/08

Abstract: Apparatuses, systems, and techniques to perform a language-guided distributional tree search based at least in part on a natural language task. In at least one embodiment, a tree search is performed using one or more neural networks to determine an action to be performed by an autonomous agent.

公开(公告)号：US11893468B2

公开(公告)日：2024-02-06

申请号：US16931211

申请日：2020-07-16

Applicant: NVIDIA Corporation

Inventor： Yu-Wei Chao ,  De-An Huang ,  Christopher Jason Paxton ,  Animesh Garg ,  Dieter Fox

IPC: G06N3/008 ,  G06N20/00

CPC classification number: G06N3/008 ,  G06N20/00

Abstract: Apparatuses, systems, and techniques to identify a goal of a demonstration. In at least one embodiment, video data of a demonstration is analyzed to identify a goal. Object trajectories identified in the video data are analyzed with respect to a task predicate satisfied by a respective object trajectory, and with respect to motion predicate. Analysis of the trajectory with respect to the motion predicate is used to assess intentionality of a trajectory with respect to the goal.

公开(公告)号：US20230234233A1

公开(公告)日：2023-07-27

申请号：US17585394

申请日：2022-01-26

Applicant: NVIDIA Corporation

Inventor： Ankit Goyal ,  Arsalan Mousavian ,  Christopher Jason Paxton ,  Yu-Wei Chao ,  Dieter Fox

IPC: B25J9/16 ,  G06T7/70 ,  G06T7/50 ,  G06T7/269 ,  G06T7/10 ,  B25J13/08 ,  G05B13/02

CPC classification number: B25J9/1697 ,  G06T7/70 ,  G06T7/50 ,  G06T7/269 ,  G06T7/10 ,  B25J13/08 ,  G05B13/027 ,  G06T2207/10024 ,  G06T2207/20084 ,  B60W60/001

Abstract: Apparatuses, systems, and techniques to place one or more objects in a location and orientation. In at least one embodiment, one or more circuits are to use one or more neural networks to cause one or more autonomous devices to place one or more objects in a location and orientation based, at least in part, on one or more images of the location and orientation.

公开(公告)号：US20230145208A1

公开(公告)日：2023-05-11

申请号：US17982401

申请日：2022-11-07

Applicant: NVIDIA Corporation

Inventor： Andreea Bobu ,  Balakumar Sundaralingam ,  Christopher Jason Paxton ,  Maya Cakmak ,  Wei Yang ,  Yu-Wei Chao ,  Dieter Fox

IPC: G06N20/00 ,  G06N5/04

CPC classification number: G06N20/00 ,  G06N5/04

Abstract: Apparatuses, systems, and techniques to train a machine learning model. In at least one embodiment, a first machine learning model is trained to infer a concept based on first information, training data is labeled using the first machine learning model, and a second machine learning model is trained to infer the concept using the labeled training data.

公开(公告)号：US12223949B2

公开(公告)日：2025-02-11

申请号：US17930349

申请日：2022-09-07

Applicant: NVIDIA Corporation

Inventor： Christopher Jason Paxton ,  Weiyu Liu ,  Tucker Ryer Hermans ,  Dieter Fox

IPC: G06F40/30 ,  B25J13/00 ,  G10L15/18 ,  G10L15/22

Abstract: A robotic system is provided for performing rearrangement tasks guided by a natural language instruction. The system can include a number of neural networks used to determine a selected rearrangement of the objects in accordance with the natural language instruction. A target object predictor network processes a point cloud of the scene and the natural language instruction to identify a set of query objects that are to-be-rearranged. A language conditioned prior network processes the point cloud, natural language instruction, and the set of query objects to sample a distribution of rearrangements to generate a number of sets of pose offsets for the set of query objects. A discriminator network then processes the samples to generate scores for the samples. The samples may be refined until a score for at least one of the sample generated by the discriminator network is above a threshold value.

公开(公告)号：US20230294277A1

公开(公告)日：2023-09-21

申请号：US17854730

申请日：2022-06-30

Applicant: Nvidia Corporation

Inventor： Wei Yang ,  Balakumar Sundaralingam ,  Christopher Jason Paxton ,  Maya Cakmak ,  Yu-Wei Chao ,  Dieter Fox ,  Iretiayo Akinola

IPC: B25J9/16 ,  G05B19/4155

CPC classification number: B25J9/1612 ,  G05B19/4155 ,  B25J9/1666 ,  B25J9/1605 ,  G05B2219/50391 ,  G05B2219/40269

Abstract: Approaches presented herein provide for predictive control of a robot or automated assembly in performing a specific task. A task to be performed may depend on the location and orientation of the robot performing that task. A predictive control system can determine a state of a physical environment at each of a series of time steps, and can select an appropriate location and orientation at each of those time steps. At individual time steps, an optimization process can determine a sequence of future motions or accelerations to be taken that comply with one or more constraints on that motion. For example, at individual time steps, a respective action in the sequence may be performed, then another motion sequence predicted for a next time step, which can help drive robot motion based upon predicted future motion and allow for quick reactions.

公开(公告)号：US20230202031A1

公开(公告)日：2023-06-29

申请号：US18116118

申请日：2023-03-01

Applicant: NVIDIA Corporation

Inventor： Wei Yang ,  Christopher Jason Paxton ,  Yu-Wei Chao ,  Dieter Fox

IPC: B25J9/16 ,  G06T7/50 ,  G06V20/30 ,  G06V20/64 ,  G06V40/10

CPC classification number: B25J9/1612 ,  G06T7/50 ,  G06V20/30 ,  G06V20/64 ,  G06V40/107 ,  G06T2207/10028 ,  B25J9/1697 ,  B25J9/16

Abstract: A robotic control system directs a robot to take an object from a human grasp by obtaining an image of a human hand holding an object, estimating the pose of the human hand and the object, and determining a grasp pose for the robot that will not interfere with the human hand. In at least one example, a depth camera is used to obtain a point cloud of the human hand holding the object. The point cloud is provided to a deep network that is trained to generate a grasp pose for a robotic gripper that can take the object from the human's hand without pinching or touching the human's fingers.

公开(公告)号：US20230073154A1

公开(公告)日：2023-03-09

申请号：US17930349

申请日：2022-09-07

Applicant: NVIDIA Corporation

Inventor： Christopher Jason Paxton ,  Weiyu Liu ,  Tucker Ryer Hermans ,  Dieter Fox

IPC: G10L15/18 ,  B25J13/00 ,  G10L15/22

Abstract: A robotic system is provided for performing rearrangement tasks guided by a natural language instruction. The system can include a number of neural networks used to determine a selected rearrangement of the objects in accordance with the natural language instruction. A target object predictor network processes a point cloud of the scene and the natural language instruction to identify a set of query objects that are to-be-rearranged. A language conditioned prior network processes the point cloud, natural language instruction, and the set of query objects to sample a distribution of rearrangements to generate a number of sets of pose offsets for the set of query objects. A discriminator network then processes the samples to generate scores for the samples. The samples may be refined until a score for at least one of the sample generated by the discriminator network is above a threshold value.

公开(公告)号：US20230297074A1

公开(公告)日：2023-09-21

申请号：US17697566

申请日：2022-03-17

Applicant: Nvidia Corporation

Inventor： Christopher Jason Paxton ,  Shagan Sah ,  Ratin Kumar ,  Dieter Fox

IPC: G05B19/4155 ,  B25J13/00 ,  B25J9/16 ,  B25J13/08

CPC classification number: G05B19/4155 ,  B25J13/003 ,  B25J9/1661 ,  B25J13/08 ,  G05B2219/50391 ,  G05B2219/40269

Abstract: Approaches provide for performance of a complex (e.g., compound) task that may involve multiple discrete tasks not obvious from an instruction to perform the complex task. A set of conditions for an environment can be determined using captured image data, and the instruction analyzed to determine a set of final conditions to exist in the environment after performance of the instruction. These initial and end conditions are used to determine a sequence of discrete tasks to be performed to cause a robot or automated device to perform the instruction. This can involve use of a symbolic or visual planner in at least some embodiments, as well as a search of possible sequences of actions available for the robot or automated device. A robot can be caused to perform the sequence of discrete tasks, and feedback provided such that the sequence of tasks can be modified as appropriate.

公开(公告)号：US20230271330A1

公开(公告)日：2023-08-31

申请号：US18055569

申请日：2022-11-15

Applicant: Nvidia Corporation

Inventor： Balakumar Sundaralingam ,  Pratyusha Sharma ,  Christopher Jason Paxton ,  Valts Blukis ,  Tucker Hermans ,  Dieter Fox

IPC: B25J13/00 ,  B25J9/16 ,  B25J19/02

CPC classification number: B25J13/003 ,  B25J9/1664 ,  B25J9/163 ,  B25J9/161 ,  B25J19/023

Abstract: Approaches presented herein provide for a framework to integrate human provided feedback in natural language to update a robot planning cost or value. The natural language feedback may be modeled as a cost or value associated with completing a task assigned to the robot. This cost or value may then be added to an initial task cost or value to update one or more actions to be performed by the robot. The framework can be applied to both real work and simulated environments where the robot may receive instructions, in natural language, that either provide a goal, modify an existing goal, or provide constraints to actions to achieve an existing goal.