公开(公告)号：US12223949B2

公开(公告)日：2025-02-11

申请号：US17930349

申请日：2022-09-07

Applicant: NVIDIA Corporation

Inventor： Christopher Jason Paxton ,  Weiyu Liu ,  Tucker Ryer Hermans ,  Dieter Fox

IPC: G06F40/30 ,  B25J13/00 ,  G10L15/18 ,  G10L15/22

Abstract: A robotic system is provided for performing rearrangement tasks guided by a natural language instruction. The system can include a number of neural networks used to determine a selected rearrangement of the objects in accordance with the natural language instruction. A target object predictor network processes a point cloud of the scene and the natural language instruction to identify a set of query objects that are to-be-rearranged. A language conditioned prior network processes the point cloud, natural language instruction, and the set of query objects to sample a distribution of rearrangements to generate a number of sets of pose offsets for the set of query objects. A discriminator network then processes the samples to generate scores for the samples. The samples may be refined until a score for at least one of the sample generated by the discriminator network is above a threshold value.

公开(公告)号：US20230073154A1

公开(公告)日：2023-03-09

申请号：US17930349

申请日：2022-09-07

Applicant: NVIDIA Corporation

Inventor： Christopher Jason Paxton ,  Weiyu Liu ,  Tucker Ryer Hermans ,  Dieter Fox

IPC: G10L15/18 ,  B25J13/00 ,  G10L15/22

Abstract: A robotic system is provided for performing rearrangement tasks guided by a natural language instruction. The system can include a number of neural networks used to determine a selected rearrangement of the objects in accordance with the natural language instruction. A target object predictor network processes a point cloud of the scene and the natural language instruction to identify a set of query objects that are to-be-rearranged. A language conditioned prior network processes the point cloud, natural language instruction, and the set of query objects to sample a distribution of rearrangements to generate a number of sets of pose offsets for the set of query objects. A discriminator network then processes the samples to generate scores for the samples. The samples may be refined until a score for at least one of the sample generated by the discriminator network is above a threshold value.