公开(公告)号：US20240371082A1

公开(公告)日：2024-11-07

申请号：US18772058

申请日：2024-07-12

Applicant: NVIDIA Corporation

Inventor： Ankit Goyal ,  Valts Blukis ,  Jie Xu ,  Yijie Guo ,  Yu-Wei Chao ,  Dieter Fox

IPC: G06T15/20 ,  B25J9/16 ,  G06T7/73 ,  G06T19/20

Abstract: In various examples, an autonomous system may use a multi-stage process to solve three-dimensional (3D) manipulation tasks from a minimal number of demonstrations and predict key-frame poses with higher precision. In a first stage of the process, for example, the disclosed systems and methods may predict an area of interest in an environment using a virtual environment. The area of interest may correspond to a predicted location of an object in the environment, such as an object that an autonomous machine is instructed to manipulate. In a second stage, the systems may magnify the area of interest and render images of the virtual environment using a 3D representation of the environment that magnifies the area of interest. The systems may then use the rendered images to make predictions related to key-frame poses associated with a future (e.g., next) state of the autonomous machine.

公开(公告)号：US20240261971A1

公开(公告)日：2024-08-08

申请号：US18232217

申请日：2023-08-09

Applicant: NVIDIA Corporation

Inventor： Yuzhe Qin ,  Wei Yang ,  Yu-Wei Chao ,  Dieter Fox

IPC: B25J9/16 ,  B25J19/02 ,  G06T7/50 ,  G06T7/70 ,  G06V10/82 ,  G06V40/10

CPC classification number: B25J9/1689 ,  B25J9/1697 ,  B25J19/023 ,  G06T7/50 ,  G06T7/70 ,  G06V10/82 ,  G06V40/10 ,  G06T2207/10028 ,  G06T2207/20084

Abstract: Apparatuses, systems, and techniques to generate control commands. In at least one embodiment, control commands are generated based on, for example, one or more images depicting a hand.

公开(公告)号：US20240177392A1

公开(公告)日：2024-05-30

申请号：US18126900

申请日：2023-03-27

Applicant: NVIDIA Corporation

Inventor： Adithyavairavan Murali ,  Arsalan Mousavian ,  Clemens Eppner ,  Adam Fishman ,  Dieter Fox

IPC: G06T15/00 ,  B25J9/16 ,  G06T9/00

CPC classification number: G06T15/00 ,  B25J9/1697 ,  G06T9/001 ,  G06T2210/21

Abstract: One common robotic task is the rearrangement of physical objects situated in an environment. This typically involves a robot manipulator picking up a target object and placing the target object in some target location, such as a shelf, cabinet or cubby, and requires the skills of picking, placing and generating complex collision-free motions, oftentimes in a cluttered environment. The present disclosure provides collision detection for object rearrangement using a three-dimensional (3D) scene representation.

公开(公告)号：US11670001B2

公开(公告)日：2023-06-06

申请号：US16416075

申请日：2019-05-17

Applicant: NVIDIA Corporation

Inventor： Arsalan Mousavian ,  Yu Xiang ,  Dieter Fox

IPC: G06T7/73 ,  G06T11/20 ,  G06V20/58 ,  G06N3/04 ,  G06N3/08

CPC classification number: G06V20/58 ,  G06N3/0454 ,  G06N3/08 ,  G06T7/75 ,  G06T11/20 ,  G06T2207/20084 ,  G06T2210/12

Abstract: In an embodiment, a system provides object tracking and 6D pose estimations to a robot that performs different tasks such as manipulation and navigation. In an embodiment the 6D object pose is determined using a Rao-Blackwellized particle filtering framework, where the 3-D rotation and the 3-D translation of the object is decoupled. In an embodiment, the system provides the 3-D translation of an object along with a full distribution over the 3-D rotation. In an embodiment, the 3-D rotation is determined by discretizing the rotation space, and training an autoencoder network to construct a codebook of feature embeddings for the discretized rotations. In an embodiment, the system is able to track objects with arbitrary symmetries while also maintaining adequate posterior distributions.

公开(公告)号：US11645492B2

公开(公告)日：2023-05-09

申请号：US16860486

申请日：2020-04-28

Applicant: NVIDIA Corporation

Inventor： Alexander Conrad Lambert ,  Adam Harper Fishman ,  Dieter Fox ,  Byron Boots ,  Fabio Tozeto Ramos

IPC: G06F17/18 ,  G06N3/04 ,  G05D1/00 ,  G06K9/62 ,  G06N3/063 ,  G06N5/04

CPC classification number: G06N3/0472 ,  G05D1/0088 ,  G06F17/18 ,  G06K9/6256 ,  G06N3/063 ,  G06N5/04

Abstract: Apparatuses, systems, and techniques to infer a sequence of actions to perform using one or more neural networks trained, at least in part, by optimizing a probability distribution function using a cost function, wherein the probability distribution represents different sequences of actions that can be performed. In at least one embodiment, a model predictive control problem is formulated as a Bayesian inference task to infer a set of solutions.

公开(公告)号：US20220374723A1

公开(公告)日：2022-11-24

申请号：US17316564

申请日：2021-05-10

Applicant: NVIDIA Corporation

Inventor： Valts Blukis ,  Christopher Jason Paxton ,  Animesh Garg ,  Dieter Fox

IPC: G06N5/00 ,  G06F16/332 ,  G06K9/62 ,  G06N3/08

Abstract: Apparatuses, systems, and techniques to perform a language-guided distributional tree search based at least in part on a natural language task. In at least one embodiment, a tree search is performed using one or more neural networks to determine an action to be performed by an autonomous agent.

公开(公告)号：US20220288783A1

公开(公告)日：2022-09-15

申请号：US17198082

申请日：2021-03-10

Applicant: NVIDIA Corporation

Inventor： Martin Sundermeyer ,  Arsalan Mousavian ,  Dieter Fox

IPC: B25J9/16 ,  G06N3/08 ,  G06T7/55

Abstract: Apparatuses, systems, and techniques to grasp objects with a robot. In at least one embodiment, a neural network is trained to determine a grasp pose of an object within a cluttered scene using a point cloud generated by a depth camera.

公开(公告)号：US20210158561A1

公开(公告)日：2021-05-27

申请号：US16922629

申请日：2020-07-07

Applicant: NVIDIA Corporation

Inventor： Keunhong Park ,  Arsalan Mousavian ,  Yu Xiang ,  Dieter Fox

IPC: G06T7/70 ,  G06T15/08 ,  G06T15/20 ,  G06T3/00 ,  B25J9/16

Abstract: Apparatuses, systems, and techniques estimate a pose of an object based on images generated from a combined image volume. In at least one embodiment, the combined image volume is obtained from a plurality of image volumes generated based on a plurality of images of an object.

公开(公告)号：US20210146531A1

公开(公告)日：2021-05-20

申请号：US16780465

申请日：2020-02-03

Applicant: NVIDIA Corporation

Inventor： Jonathan Tremblay ,  Dieter Fox ,  Michelle Lee ,  Carlos Florensa ,  Nathan Donald Ratliff ,  Animesh Garg ,  Fabio Tozeto Ramos

IPC: B25J9/16 ,  G05B13/02 ,  G06N3/08 ,  G05B13/04 ,  G06N5/04 ,  G06N20/00

Abstract: A robot is controlled using a combination of model-based and model-free control methods. In some examples, the model-based method uses a physical model of the environment around the robot to guide the robot. The physical model is oriented using a perception system such as a camera. Characteristics of the perception system may be are used to determine an uncertainty for the model. Based at least in part on this uncertainty, the system transitions from the model-based method to a model-free method where, in some embodiments, information provided directly from the perception system is used to direct the robot without reliance on the physical model.

公开(公告)号：US12299800B2

公开(公告)日：2025-05-13

申请号：US18126900

申请日：2023-03-27

Applicant: NVIDIA Corporation

Inventor： Adithyavairavan Murali ,  Arsalan Mousavian ,  Clemens Eppner ,  Adam Fishman ,  Dieter Fox

IPC: G06T15/00 ,  B25J9/16 ,  G06T9/00

Abstract: One common robotic task is the rearrangement of physical objects situated in an environment. This typically involves a robot manipulator picking up a target object and placing the target object in some target location, such as a shelf, cabinet or cubby, and requires the skills of picking, placing and generating complex collision-free motions, oftentimes in a cluttered environment. The present disclosure provides collision detection for object rearrangement using a three-dimensional (3D) scene representation.