公开(公告)号：US20250073901A1

公开(公告)日：2025-03-06

申请号：US18239601

申请日：2023-08-29

Applicant: NVIDIA Corporation

Inventor： Ajay Uday Mandlekar ,  Soroush Nasiriany ,  Bowen Wen ,  Iretiayo Akinola ,  Yashraj Shyam Narang ,  Linxi Fan ,  Yuke Zhu ,  Dieter Fox

IPC: B25J9/16 ,  B25J19/02

Abstract: Apparatuses, systems, and techniques to generate data to train a robotic device to perform tasks. In at least one embodiment, one or more first videos of a robotic device performing a task is used to generate one or more second videos of the robotic device performing the task differently than depicted in the one or more first videos.

公开(公告)号：US12202147B2

公开(公告)日：2025-01-21

申请号：US17695756

申请日：2022-03-15

Applicant: NVIDIA CORPORATION

Inventor： Ankur Handa ,  Iretiayo Akinola ,  Dieter Fox ,  Yashraj Shyam Narang

IPC: B25J9/16

Abstract: A technique for training a neural network, including generating a plurality of input vectors based on a first plurality of task demonstrations associated with a first robot performing a first task in a simulated environment, wherein each input vector included in the plurality of input vectors specifies a sequence of poses of an end-effector of the first robot, and training the neural network to generate a plurality of output vectors based on the plurality of input vectors. Another technique for generating a task demonstration, including generating a simulated environment that includes a robot and at least one object, causing the robot to at least partially perform a task associated with the at least one object within the simulated environment based on a first output vector generated by a trained neural network, and recording demonstration data of the robot at least partially performing the task within the simulated environment.

公开(公告)号：US20250010475A1

公开(公告)日：2025-01-09

申请号：US18888045

申请日：2024-09-17

Applicant: NVIDIA Corporation

Inventor： Carolyn Linjon Chen ,  Yashraj Shyam Narang ,  Fabio Tozeto Ramos ,  Dieter Fox

IPC: B25J9/16 ,  B25J11/00 ,  G01N15/00 ,  G06F17/18 ,  G06F18/214 ,  G06F30/27 ,  G06N3/08 ,  G06N7/01 ,  G06T7/00 ,  G06T7/77 ,  G06V10/764 ,  G06V10/82 ,  G06V20/56 ,  G06V20/64

Abstract: Apparatuses, systems, and techniques to identify at least one physical characteristic of materials from computer simulations of manipulations of materials. In at least one embodiment, physical characteristics are determined by comparing measured statistics of observed manipulations to simulations of manipulations using a simulator trained with a likelihood-free inference engine.

公开(公告)号：US12138805B2

公开(公告)日：2024-11-12

申请号：US17198082

申请日：2021-03-10

Applicant: NVIDIA Corporation

Inventor： Martin Sundermeyer ,  Arsalan Mousavian ,  Dieter Fox

IPC: B25J9/16 ,  G06N3/08 ,  G06T7/55

Abstract: Apparatuses, systems, and techniques to grasp objects with a robot. In at least one embodiment, a neural network is trained to determine a grasp pose of an object within a cluttered scene using a point cloud generated by a depth camera.

公开(公告)号：US12109701B2

公开(公告)日：2024-10-08

申请号：US16780465

申请日：2020-02-03

Applicant: NVIDIA Corporation

Inventor： Jonathan Tremblay ,  Dieter Fox ,  Michelle Lee ,  Carlos Florensa ,  Nathan Donald Ratliff ,  Animesh Garg ,  Fabio Tozeto Ramos

IPC: B25J9/16 ,  G05B13/02 ,  G05B13/04 ,  G06N3/08 ,  G06N5/046 ,  G06N20/00

CPC classification number: B25J9/163 ,  B25J9/1661 ,  B25J9/1664 ,  B25J9/1697 ,  G05B13/027 ,  G05B13/04 ,  G06N3/08 ,  G06N5/046 ,  G06N20/00

Abstract: A robot is controlled using a combination of model-based and model-free control methods. In some examples, the model-based method uses a physical model of the environment around the robot to guide the robot. The physical model is oriented using a perception system such as a camera. Characteristics of the perception system may be are used to determine an uncertainty for the model. Based at least in part on this uncertainty, the system transitions from the model-based method to a model-free method where, in some embodiments, information provided directly from the perception system is used to direct the robot without reliance on the physical model.

公开(公告)号：US20240273810A1

公开(公告)日：2024-08-15

申请号：US18430113

申请日：2024-02-01

Applicant: NVIDIA Corporation

Inventor： Ankit Goyal ,  Jie Xu ,  Yijie Guo ,  Valts Blukis ,  Yu-Wei Chao ,  Dieter Fox

IPC: G06T15/10 ,  G05D1/243 ,  G05D101/15 ,  G06T7/55

CPC classification number: G06T15/10 ,  G05D1/2435 ,  G06T7/55 ,  G05D2101/15 ,  G06T2207/20084

Abstract: In various examples, a machine may generate, using sensor data capturing one or more views of an environment, a virtual environment including a 3D representation of the environment. The machine may render, using one or more virtual sensors in the virtual environment, one or more images of the 3D representation of the environment. The machine may apply the one or more images to one or more machine learning models (MLMs) trained to generate one or more predictions corresponding to the environment. The machine may perform one or more control operations based at least on the one or more predictions generated using the one or more MLMs.

公开(公告)号：US20240096074A1

公开(公告)日：2024-03-21

申请号：US17581550

申请日：2022-01-21

Applicant: Nvidia Corporation

Inventor： Brian Okorn ,  Arsalan Mousavian ,  Lucas Manuelli ,  Dieter Fox

IPC: G06V10/82 ,  G06V10/26 ,  G06V10/77

CPC classification number: G06V10/82 ,  G06V10/26 ,  G06V10/7715

Abstract: Apparatuses, systems, and techniques are presented to identify one or more objects. In at least one embodiment, one or more neural networks can be used to identify one or more objects based, at least in part, on one or more descriptors of one or more segments of the one or more objects.

公开(公告)号：US11724401B2

公开(公告)日：2023-08-15

申请号：US16914105

申请日：2020-06-26

Applicant: NVIDIA Corporation

Inventor： Arsalan Mousavian ,  Clemens Eppner ,  Dieter Fox ,  Adithyavairavan Murali

IPC: G06K9/00 ,  B25J9/16 ,  G05B19/4155 ,  B25J13/08 ,  G06N3/08 ,  G06T7/50 ,  G06T7/10 ,  G06T7/70 ,  G05B19/402

CPC classification number: B25J9/1697 ,  B25J9/161 ,  B25J9/1612 ,  B25J9/1666 ,  B25J13/08 ,  G05B19/402 ,  G05B19/4155 ,  G06N3/08 ,  G06T7/10 ,  G06T7/50 ,  G06T7/70 ,  G05B2219/40269 ,  G06T2207/10028 ,  G06T2207/20084 ,  G06T2207/20132 ,  G06T2207/30244

Abstract: Apparatuses, systems, and techniques determine a set of grasp poses that would allow a robot to successfully grasp an object that is proximate to at least one additional object. In at least one embodiment, the set of grasp poses is modified based on a determination that at least one of the grasp poses in the set of grasp poses would interfere with at least one additional object that is proximate to the object.

公开(公告)号：US20210125036A1

公开(公告)日：2021-04-29

申请号：US16667708

申请日：2019-10-29

Applicant: NVIDIA Corporation

Inventor： Jonathan Tremblay ,  Ming-Yu Liu ,  Dieter Fox ,  Philip Ammirato

IPC: G06N3/04 ,  G10L15/16 ,  G06K9/46 ,  G06K9/66 ,  G05D1/00 ,  G06N3/08

Abstract: Apparatuses, systems, and techniques to determine orientation of an objects in an image. In at least one embodiment, images are processed using a neural network trained to determine orientation of an object.

公开(公告)号：US20210122045A1

公开(公告)日：2021-04-29

申请号：US16863111

申请日：2020-04-30

Applicant: NVIDIA Corporation

Inventor： Ankur Handa ,  Karl Van Wyk ,  Viktor Makoviichuk ,  Dieter Fox

IPC: B25J9/16 ,  G05B17/02 ,  B25J13/08 ,  G06T7/73

Abstract: Apparatuses, systems, and techniques are described that estimate the pose of an object while the object is being manipulated by a robotic appendage. In at least one embodiment, a sample-based optimization algorithm tracks in-hand object poses during manipulation via contact feedback and a GPU-accelerated robotic simulation is developed. In at least one embodiment, parallel simulations concurrently model object pose changes that may be caused by complex contact dynamics. In at least one embodiment, the optimization algorithm tunes simulation parameters during object pose tracking to further improve tracking performance. In various embodiments, real-world contact sensing may be improved by utilizing vision in-the-loop.