-
公开(公告)号:US11195041B2
公开(公告)日:2021-12-07
申请号:US16864591
申请日:2020-05-01
Applicant: X Development LLC
Inventor: Kurt Konolige , Nareshkumar Rajkumar , Stefan Hinterstoisser
Abstract: Methods and apparatus related to generating a model for an object encountered by a robot in its environment, where the object is one that the robot is unable to recognize utilizing existing models associated with the robot. The model is generated based on vision sensor data that captures the object from multiple vantages and that is captured by a vision sensor associated with the robot, such as a vision sensor coupled to the robot. The model may be provided for use by the robot in detecting the object and/or for use in estimating the pose of the object.
-
公开(公告)号:US11192250B1
公开(公告)日:2021-12-07
申请号:US16665273
申请日:2019-10-28
Applicant: X Development LLC
Inventor: Stefan Hinterstoisser
Abstract: Methods, apparatus, and computer readable media that are related to 3D object detection and pose determination and that may optionally increase the robustness and/or efficiency of the 3D object recognition and pose determination. Some implementations are generally directed to techniques for generating an object model of an object based on model point cloud data of the object. Some implementations of the present disclosure are additionally and/or alternatively directed to techniques for application of acquired 3D scene point cloud data to a stored object model of an object to detect the object and/or determine the pose of the object.
-
公开(公告)号:US10773382B2
公开(公告)日:2020-09-15
申请号:US15913212
申请日:2018-03-06
Applicant: X Development LLC
Inventor: Yunfei Bai , Kuan Fang , Stefan Hinterstoisser , Mrinal Kalakrishnan
IPC: B25J9/16
Abstract: Implementations are directed to training a machine learning model that, once trained, is used in performance of robotic grasping and/or other manipulation task(s) by a robot. The model can be trained using simulated training examples that are based on simulated data that is based on simulated robot(s) attempting simulated manipulations of various simulated objects. At least portions of the model can also be trained based on real training examples that are based on data from real-world physical robots attempting manipulations of various objects. The simulated training examples can be utilized to train the model to predict an output that can be utilized in a particular task—and the real training examples used to adapt at least a portion of the model to the real-world domain can be tailored to a distinct task. In some implementations, domain-adversarial similarity losses are determined during training, and utilized to regularize at least portion(s) of the model.
-
公开(公告)号:US20200265260A1
公开(公告)日:2020-08-20
申请号:US16864591
申请日:2020-05-01
Applicant: X Development LLC
Inventor: Kurt Konolige , Nareshkumar Rajkumar , Stefan Hinterstoisser
Abstract: Methods and apparatus related to generating a model for an object encountered by a robot in its environment, where the object is one that the robot is unable to recognize utilizing existing models associated with the robot. The model is generated based on vision sensor data that captures the object from multiple vantages and that is captured by a vision sensor associated with the robot, such as a vision sensor coupled to the robot. The model may be provided for use by the robot in detecting the object and/or for use in estimating the pose of the object.
-
公开(公告)号:US20200078938A1
公开(公告)日:2020-03-12
申请号:US16687106
申请日:2019-11-18
Applicant: X Development LLC
Inventor: Gary Bradski , Steve Croft , Kurt Konolige , Ethan Rublee , Troy Straszheim , John Zevenbergen , Stefan Hinterstoisser , Hauke Strasdat
IPC: B25J9/16 , G06T7/55 , G06K9/62 , G06K9/32 , G06K9/00 , B25J19/00 , G06T19/00 , B65G41/00 , B25J5/00 , B25J9/00 , H04N5/33 , G06T7/60 , G06K9/52 , G06K9/46 , G01B11/25 , B65G47/50 , B65G47/46 , B25J19/02 , H04N13/239 , G06T17/00 , G06T7/13 , G06T7/593 , G06T7/529
Abstract: Example embodiments may relate to methods and systems for selecting a grasp point on an object. In particular, a robotic manipulator may identify characteristics of a physical object within a physical environment. Based on the identified characteristics, the robotic manipulator may determine potential grasp points on the physical object corresponding to points at which a gripper attached to the robotic manipulator is operable to grip the physical object. Subsequently, the robotic manipulator may determine a motion path for the gripper to follow in order to move the physical object to a drop-off location for the physical object and then select a grasp point, from the potential grasp points, based on the determined motion path. After selecting the grasp point, the robotic manipulator may grip the physical object at the selected grasp point with the gripper and move the physical object through the determined motion path to the drop-off location.
-
公开(公告)号:US10518410B2
公开(公告)日:2019-12-31
申请号:US15968323
申请日:2018-05-01
Applicant: X Development LLC
Inventor: Gary Bradski , Steve Croft , Kurt Konolige , Ethan Rublee , Troy Straszheim , John Zevenbergen , Stefan Hinterstoisser , Hauke Strasdat
IPC: G05B19/18 , G05B19/04 , B25J9/16 , G06T7/529 , G06T7/593 , G06T7/13 , G06T17/00 , G06K9/00 , H04N13/239 , B25J19/02 , B65G47/46 , B65G47/50 , G01B11/25 , G06K9/46 , G06K9/52 , G06T7/60 , H04N5/33 , B25J9/00 , B25J5/00 , B65G41/00 , G06T19/00 , B25J19/00 , G06K9/32 , G06K9/62 , B65G61/00 , B65H67/06 , H04N13/00
Abstract: Example embodiments may relate to methods and systems for selecting a grasp point on an object. In particular, a robotic manipulator may identify characteristics of a physical object within a physical environment. Based on the identified characteristics, the robotic manipulator may determine potential grasp points on the physical object corresponding to points at which a gripper attached to the robotic manipulator is operable to grip the physical object. Subsequently, the robotic manipulator may determine a motion path for the gripper to follow in order to move the physical object to a drop-off location for the physical object and then select a grasp point, from the potential grasp points, based on the determined motion path. After selecting the grasp point, the robotic manipulator may grip the physical object at the selected grasp point with the gripper and move the physical object through the determined motion path to the drop-off location.
-
公开(公告)号:US10417781B1
公开(公告)日:2019-09-17
申请号:US15396105
申请日:2016-12-30
Applicant: X Development LLC
Inventor: Kurt Konolige , Nareshkumar Rajkumar , Stefan Hinterstoisser , Paul Wohlhart
Abstract: Methods for annotating objects within image frames are disclosed. Information is obtained that represents a camera pose relative to a scene. The camera pose includes a position and a location of the camera relative to the scene. Data is obtained that represents multiple images, including a first image and a plurality of other images, being captured from different angles by the camera relative to the scene. A 3D pose of the object of interest is identified with respect to the camera pose in at least the first image. A 3D bounding region for the object of interest in the first image is defined, which indicates a volume that includes the object of interest. A location and orientation of the object of interest is determined in the other images based on the defined 3D bounding region of the object of interest and the camera pose in the other images.
-
公开(公告)号:US20180300550A1
公开(公告)日:2018-10-18
申请号:US16014311
申请日:2018-06-21
Applicant: X Development LLC
Inventor: Nareshkumar Rajkumar , Stefan Hinterstoisser
CPC classification number: G06K9/00664 , G06K9/00979 , G06K9/6296 , G06T7/33 , G06T7/73 , G06T2207/10021
Abstract: Methods, apparatus, systems, and computer-readable media are provided for delegating object type and/or pose detection to a plurality of “targeted object recognition modules.” In some implementations, a method may be provided that includes: operating an object recognition client to facilitate object recognition for a robot; receiving, by the object recognition client, sensor data indicative of an observed object in an environment; providing, by the object recognition client, to each of a plurality of remotely-hosted targeted object recognition modules, data indicative of the observed object; receiving, by the object recognition client, from one or more of the plurality of targeted object recognition modules, one or more inferences about an object type or pose of the observed object; and determining, by the object recognition client, information about the observed object, such as its object type and/or pose, based on the one or more inferences.
-
公开(公告)号:US20180243904A1
公开(公告)日:2018-08-30
申请号:US15968323
申请日:2018-05-01
Applicant: X Development LLC
Inventor: Gary Bradski , Steve Croft , Kurt Konolige , Ethan Rublee , Troy Straszheim , John Zevenbergen , Stefan Hinterstoisser , Hauke Strasdat
Abstract: Example embodiments may relate to methods and systems for selecting a grasp point on an object. In particular, a robotic manipulator may identify characteristics of a physical object within a physical environment. Based on the identified characteristics, the robotic manipulator may determine potential grasp points on the physical object corresponding to points at which a gripper attached to the robotic manipulator is operable to grip the physical object. Subsequently, the robotic manipulator may determine a motion path for the gripper to follow in order to move the physical object to a drop-off location for the physical object and then select a grasp point, from the potential grasp points, based on the determined motion path. After selecting the grasp point, the robotic manipulator may grip the physical object at the selected grasp point with the gripper and move the physical object through the determined motion path to the drop-off location.
-
公开(公告)号:US10025984B2
公开(公告)日:2018-07-17
申请号:US15212967
申请日:2016-07-18
Applicant: X DEVELOPMENT LLC
Inventor: Nareshkumar Rajkumar , Stefan Hinterstoisser
Abstract: Methods, apparatus, systems, and computer-readable media are provided for delegating object type and/or pose detection to a plurality of “targeted object recognition modules.” In some implementations, a method may be provided that includes: operating an object recognition client to facilitate object recognition for a robot; receiving, by the object recognition client, sensor data indicative of an observed object in an environment; providing, by the object recognition client, to each of a plurality of remotely-hosted targeted object recognition modules, data indicative of the observed object; receiving, by the object recognition client, from one or more of the plurality of targeted object recognition modules, one or more inferences about an object type or pose of the observed object; and determining, by the object recognition client, information about the observed object, such as its object type and/or pose, based on the one or more inferences.
-
-
-
-
-
-
-
-
-