公开(公告)号：US10521930B1

公开(公告)日：2019-12-31

申请号：US16506695

申请日：2019-07-09

Applicant: MUJIN, INC.

Inventor： Russell Islam ,  Xutao Ye

IPC: G06T7/80 ,  H04N13/246 ,  G06T7/73

Abstract: A system and method for performing automatic camera calibration is presented. The system communicates with a first camera and a second camera, wherein a transparent platform is disposed between the two cameras. When a 3D calibration pattern is disposed on the platform, the system receives a first set of calibration images from the first camera, and a second set of calibration images from the second camera. The system determines, based on the first set of calibration images, a first set of coordinates for corners of the polyhedron. The system further determines, based on the second set of calibration images, a second set of coordinates for the corners. The system determines, based on the coordinates, a spatial relationship between the first camera and the second camera. The system further uses a description of the spatial relationship to generate a 3D model of an object other than the 3D calibration pattern.

公开(公告)号：US11780101B2

公开(公告)日：2023-10-10

申请号：US17313921

申请日：2021-05-06

Applicant: MUJIN, Inc.

Inventor： Rosen Diankov ,  Huan Liu ,  Xutao Ye ,  Jose Jeronimo Moreira Rodrigues ,  Yoshiki Kanemoto ,  Jinze Yu ,  Russell Islam

IPC: B25J15/06 ,  B25J9/16 ,  G06T7/13 ,  G06T7/50 ,  B25J13/08 ,  G06T7/00 ,  G06T7/66 ,  B25J9/00 ,  B65G47/91 ,  B65G61/00 ,  G06T1/00 ,  G06V10/44 ,  G06V20/10 ,  G06F18/22

CPC classification number: B25J15/0616 ,  B25J9/0093 ,  B25J9/1697 ,  B25J13/085 ,  B65G47/91 ,  B65G61/00 ,  G06F18/22 ,  G06T1/0014 ,  G06T7/001 ,  G06T7/13 ,  G06T7/50 ,  G06T7/66 ,  G06V10/443 ,  G06V20/10 ,  G05B2219/39001 ,  G05B2219/40067 ,  G06T2207/20212

Abstract: A method of operating a robotic system includes: receiving first image data representative of a package surface; identifying a pair of edges based on the first image data; determining a minimum viable region based on the pair of edges; receiving second image data representative of the package after the lift; and creating registration data based on the first and second image data.

公开(公告)号：US20220076425A1

公开(公告)日：2022-03-10

申请号：US17527012

申请日：2021-11-15

Applicant: MUJIN, Inc.

Inventor： Jinze Yu ,  Jose Jeronimo Moreira Rodrigues ,  Rosen Nikolaev Diankov ,  Xutao Ye ,  Russell Islam

IPC: G06T7/13 ,  B25J9/16 ,  B65B43/46 ,  B65G61/00 ,  G05B19/4093 ,  G06K9/00 ,  G06K9/32 ,  G06Q10/08 ,  G06T7/11 ,  G06T7/50 ,  G06T7/62 ,  G06T7/73 ,  G06T19/20

Abstract: The present disclosure relates to detecting and registering unrecognized or unregistered objects. A minimum viable range (MVR) may be derived based on inspecting image data that represents objects. The MVR may be determined to be a certain MVR or an uncertain MVR according to one or more features represented in the image data. The MVR may be used to register corresponding objects according to the certain or uncertain determination.

公开(公告)号：US11189033B2

公开(公告)日：2021-11-30

申请号：US16667840

申请日：2019-10-29

Applicant: MUJIN, Inc.

Inventor： Jinze Yu ,  Jose Jeronimo Moreira Rodrigues ,  Rosen Nikolaev Diankov ,  Xutao Ye ,  Russell Islam

IPC: G06F7/00 ,  G06T7/13 ,  B25J9/16 ,  G06T7/73 ,  G06T7/50 ,  G06T7/62 ,  B65G61/00 ,  G05B19/4093 ,  G06Q10/08 ,  G06T19/20 ,  G06T7/11 ,  B65B43/46 ,  G06K9/00 ,  G06K9/32

Abstract: The present disclosure relates to detecting and registering unrecognized or unregistered objects. A minimum viable range (MVR) may be derived based on inspecting image data that represents objects at a start location. The MVR may be determined to be a certain MVR or an uncertain MVR according to one or more features represented in the image data. The MVR may be used to register corresponding objects according to the certain or uncertain determination.

公开(公告)号：US11014241B2

公开(公告)日：2021-05-25

申请号：US16732832

申请日：2020-01-02

Applicant: MUJIN, INC.

Inventor： Russell Islam ,  Xutao Ye ,  Rosen Diankov

IPC: B25J9/16 ,  G06T7/80

Abstract: A robot control system and a method for camera calibration verification is presented. The robot control system is configured to perform a first camera calibration, and to control a robot arm to move a verification symbol to a reference location. The robot control system further receives, from a camera, a reference image of the verification symbol, and determines a reference image coordinate for the verification symbol. The robot control system further controls the robot arm to move the verification symbol to the reference location again during an idle period, receives an additional image of the verification symbol, and determines a verification image coordinate. The robot control system determines a deviation parameter value based the reference image coordinate and the verification image coordinate, and whether the deviation parameter value exceeds a defined threshold, and performs a second camera calibration if the threshold is exceeded.

公开(公告)号：US10565737B1

公开(公告)日：2020-02-18

申请号：US16506695

申请日：2019-07-09

Applicant: MUJIN, INC.

Inventor： Russell Islam ,  Xutao Ye

IPC: G06T7/80 ,  H04N13/246 ,  G06T7/73

Abstract: A system and method for performing automatic camera calibration is presented. The system communicates with a first camera and a second camera, wherein a transparent platform is disposed between the two cameras. When a 3D calibration pattern is disposed on the platform, the system receives a first set of calibration images from the first camera, and a second set of calibration images from the second camera. The system determines, based on the first set of calibration images, a first set of coordinates for corners of the polyhedron. The system further determines, based on the second set of calibration images, a second set of coordinates for the corners. The system determines, based on the coordinates, a spatial relationship between the first camera and the second camera. The system further uses a description of the spatial relationship to generate a 3D model of an object other than the 3D calibration pattern.

公开(公告)号：US10399227B1

公开(公告)日：2019-09-03

申请号：US16369630

申请日：2019-03-29

Applicant: MUJIN, INC.

Inventor： Russell Islam ,  Xutao Ye ,  Rosen Diankov

IPC: B25J9/16 ,  G06T7/80

Abstract: A robot control system and a method for camera calibration verification is presented. The robot control system is configured to perform a first camera calibration, and to control a robot arm to move a verification symbol to a reference location. The robot control system further receives, from a camera, a reference image of the verification symbol, and determines a reference image coordinate for the verification symbol. The robot control system further controls the robot arm to move the verification symbol to the reference location again during an idle period, receives an additional image of the verification symbol, and determines a verification image coordinate. The robot control system determines a deviation parameter value based the reference image coordinate and the verification image coordinate, and whether the deviation parameter value exceeds a defined threshold, and performs a second camera calibration if the threshold is exceeded.

公开(公告)号：US10369701B1

公开(公告)日：2019-08-06

申请号：US16290741

申请日：2019-03-01

Applicant: Mujin, Inc.

Inventor： Rosen Diankov ,  Huan Liu ,  Xutao Ye ,  Jose Jeronimo Moreira Rodrigues ,  Yoshiki Kanemoto ,  Jinze Yu ,  Russell Islam

IPC: B25J9/16 ,  G06T7/13 ,  G06T7/50 ,  G06K9/62 ,  B25J13/08 ,  G06T7/00

Abstract: A method of operating a package handling system includes: receiving first image data representative of a first surface image of a package surface; identifying a pair of intersecting edges for the package surface based on the first image; determining a minimum viable region based on the pair of edges, the minimum viable region for gripping and lifting the package; receiving second image data representative of the package after the lift; and creating registration data based on the third image data.

公开(公告)号：US10369698B1

公开(公告)日：2019-08-06

申请号：US16295940

申请日：2019-03-07

Applicant: MUJIN, INC.

Inventor： Russell Islam ,  Xutao Ye ,  Rosen Diankov

IPC: B25J9/16 ,  G06T7/80

Abstract: A robot control system and a method for automatic camera calibration is presented. The robot control system includes a control circuit configured to determine all corner locations of an imaginary cube that fits within a camera field of view, and determine a plurality of locations that are distributed on or throughout the imaginary cube. The control circuit is further configured to control a robot arm to move a calibration pattern to the plurality of locations, and to receive a plurality of calibration images corresponding to the plurality of locations, and to determine respective estimates of intrinsic camera parameters based on the plurality of calibration images, and to determine an estimate of a transformation function that describes a relationship between a camera coordinate system and a world coordinate system. The control circuit is further configured to control placement of the robot arm based on the estimate of the transformation function.

公开(公告)号：US12002007B2

公开(公告)日：2024-06-04

申请号：US17944117

申请日：2022-09-13

Applicant: MUJIN, Inc.

Inventor： Rosen Nikolaev Diankov ,  Huan Liu ,  Xutao Ye ,  Jose Jeronimo Moreira Rodrigues ,  Yoshiki Kanemoto ,  Jinze Yu ,  Russell Islam

IPC: G06Q10/08 ,  B25J9/16 ,  B65B43/46 ,  B65G61/00 ,  G05B19/4093 ,  G06Q10/087 ,  G06T7/11 ,  G06T7/13 ,  G06T7/50 ,  G06T7/62 ,  G06T7/73 ,  G06T19/20 ,  G06V10/20 ,  G06V10/25 ,  G06V20/64 ,  G06V10/44

CPC classification number: G06Q10/087 ,  B25J9/1676 ,  B25J9/1687 ,  B25J9/1697 ,  B65B43/46 ,  B65G61/00 ,  G05B19/40935 ,  G06T7/11 ,  G06T7/13 ,  G06T7/50 ,  G06T7/62 ,  G06T7/73 ,  G06T19/20 ,  G06V10/25 ,  G06V10/255 ,  G06V20/64 ,  G05B2219/40006 ,  G05B2219/40607 ,  G06T2200/04 ,  G06T2207/10028 ,  G06T2207/20164 ,  G06T2207/30108 ,  G06V10/44

Abstract: A system and method for operating a robotic system to scan and register unrecognized objects is disclosed. The robotic system may use an image data representative of an unrecognized object located at a start location to implement operations for transferring the unrecognized object from the start location. While implementing the operations, the robotic system may obtain additional data, including scanning results of one or more portions of the unrecognized object not included in the image data. The robotic system may use the additional data to register the unrecognized object.