公开(公告)号：US20240316779A1

公开(公告)日：2024-09-26

申请号：US18607407

申请日：2024-03-15

Applicant: MUJIN, Inc.

Inventor： Yoshiki Kanemoto ,  Shintaro Matsuoka ,  Jose Jeronimo Moreira Rodrigues ,  Kentaro Wada ,  Rosen Nikolaev Diankov ,  Puttichai Lertkultanon ,  Lei Lei ,  Yixuan Zhang ,  Xutao Ye ,  Yufan Du ,  Mingjian Liang ,  Lingping Gao ,  Xinhao Wen ,  Xu Chen

IPC: B25J9/16 ,  B25J9/00 ,  B65G67/08 ,  B65G67/20 ,  B65G67/24

CPC classification number: B25J9/1687 ,  B25J9/0093 ,  B25J9/1612 ,  B25J9/1697 ,  B65G67/08 ,  B65G67/20 ,  B65G67/24

Abstract: A robotic system may include a chassis operatively coupled to a proximal conveyor, a first segment including a first segment conveyor extending along a length of the first segment, and a gripper including a distal conveyor extending along a length of the gripper. The robotic system may further include a controller configured to operate the chassis, the conveyors, the segments, the gripper, or a combination thereof to remove and transfer objects away from a cargo loading structure, such as a cargo container.

公开(公告)号：US11967113B2

公开(公告)日：2024-04-23

申请号：US17350006

申请日：2021-06-17

Applicant: MUJIN, INC.

Inventor： Russell Islam ,  Xutao Ye

IPC: G06T7/80 ,  G06T7/73 ,  H04N13/246

CPC classification number: G06T7/85 ,  G06T7/75 ,  H04N13/246 ,  G06T2207/10012

Abstract: A system and method for performing automatic camera calibration is presented. The system communicates with a first camera and a second camera, wherein a transparent platform is disposed between the two cameras. When a 3D calibration pattern is disposed on the platform, the system receives a first set of calibration images from the first camera, and a second set of calibration images from the second camera. The system determines, based on the first set of calibration images, a first set of coordinates for corners of the polyhedron. The system further determines, based on the second set of calibration images, a second set of coordinates for the corners. The system determines, based on the coordinates, a spatial relationship between the first camera and the second camera. The system further uses a description of the spatial relationship to generate a 3D model of an object other than the 3D calibration pattern.

公开(公告)号：US20240020637A1

公开(公告)日：2024-01-18

申请号：US18303323

申请日：2023-04-19

Applicant: MUJIN, Inc.

Inventor： Jinze Yu ,  Jose Jeronimo Moreira Rodrigues ,  Rosen Nikolaev Diankov ,  Xutao Ye

IPC: G06Q10/087 ,  G06T7/13 ,  B25J9/16 ,  G06T7/73 ,  G06T7/50 ,  G06T7/62 ,  B65G61/00 ,  G05B19/4093 ,  G06T19/20 ,  G06T7/11 ,  B65B43/46 ,  G06V10/25 ,  G06V20/64 ,  G06V10/20

CPC classification number: G06Q10/087 ,  G06T7/13 ,  B25J9/1697 ,  G06T7/73 ,  G06T7/50 ,  G06T7/62 ,  B25J9/1687 ,  B65G61/00 ,  G05B19/40935 ,  G06T19/20 ,  G06T7/11 ,  B65B43/46 ,  B25J9/1676 ,  G06V10/25 ,  G06V20/64 ,  G06V10/255 ,  G06T2207/30108 ,  G06T2207/20164 ,  G06T2200/04 ,  G05B2219/40006 ,  G05B2219/40607 ,  G06T2207/10028 ,  G06V10/44

Abstract: The present disclosure relates to methods and systems for generating a verified minimum viable range (MVR) of an object. An exposed outer corner and exposed edges of an object may be identified by processing one or more image data. An initial MVR may be generated by identifying opposing parallel edges opposing the exposed edges. The initial MVR may be adjusted, and the adjusted result may be tested to generate a verified MVR.

公开(公告)号：US20230415360A1

公开(公告)日：2023-12-28

申请号：US18463233

申请日：2023-09-07

Applicant: MUJIN, Inc.

Inventor： Rosen Diankov ,  Huan Liu ,  Xutao Ye ,  Jose Jeronimo Moreira Rodrigues ,  Yoshiki Kanemoto ,  Jinze Yu ,  Russell Islam

IPC: B25J15/06 ,  B25J9/16 ,  G06T7/13 ,  G06T7/50 ,  B25J13/08 ,  G06T7/00 ,  G06T7/66 ,  B25J9/00 ,  B65G47/91 ,  B65G61/00 ,  G06T1/00 ,  G06V10/44 ,  G06V20/10 ,  G06F18/22

CPC classification number: B25J15/0616 ,  B25J9/1697 ,  G06T7/13 ,  G06T7/50 ,  B25J13/085 ,  G06T7/001 ,  G06T7/66 ,  B25J9/0093 ,  B65G47/91 ,  B65G61/00 ,  G06T1/0014 ,  G06V10/443 ,  G06V20/10 ,  G06F18/22 ,  G05B2219/40067 ,  G06T2207/20212 ,  G05B2219/39001

Abstract: A method of operating a robotic system includes: receiving first image data representative of a package surface; identifying a pair of edges based on the first image data; determining a minimum viable region based on the pair of edges; receiving second image data representative of the package after the lift; and creating registration data based on the first and second image data.

公开(公告)号：US11590656B2

公开(公告)日：2023-02-28

申请号：US17135261

申请日：2020-12-28

Applicant: MUJIN, INC.

Inventor： Russell Islam ,  Xutao Ye ,  Rosen Diankov

IPC: B25J9/16 ,  G06T7/80

Abstract: A computing system and a method for calibration verification is presented. The computing system is configured to perform a first calibration operation, and to control a robot arm to move a verification symbol to a reference location. The robot control system further receives, from a camera, a reference image of the verification symbol, and determines a reference image coordinate for the verification symbol. The robot control system further controls the robot arm to move the verification symbol to the reference location again during an idle period, receives an additional image of the verification symbol, and determines a verification image coordinate. The robot control system determines a deviation parameter value based the reference image coordinate and the verification image coordinate, and whether the deviation parameter value exceeds a defined threshold, and performs a second calibration operation if the threshold is exceeded.

公开(公告)号：US20230008540A1

公开(公告)日：2023-01-12

申请号：US17944117

申请日：2022-09-13

Applicant: MUJIN, Inc.

Inventor： Rosen Nikolaev Diankov ,  Huan Liu ,  Xutao Ye ,  Jose Jeronimo Moreira Rodrigues ,  Yoshiki Kanemoto ,  Jinze Yu ,  Russell Islam

IPC: G06T7/13 ,  B25J9/16 ,  G06Q10/08 ,  G06T7/11 ,  G06T7/50 ,  G06V10/25 ,  G06T19/20 ,  G06T7/62 ,  B65G61/00 ,  G06V20/64 ,  G05B19/4093 ,  B65B43/46 ,  G06T7/73

Abstract: A system and method for operating a robotic system to scan and register unrecognized objects is disclosed. The robotic system may use an image data representative of an unrecognized object located at a start location to implement operations for transferring the unrecognized object from the start location. While implementing the operations, the robotic system may obtain additional data, including scanning results of one or more portions of the unrecognized object not included in the image data. The robotic system may use the additional data to register the unrecognized object.

公开(公告)号：US10583560B1

公开(公告)日：2020-03-10

申请号：US16374617

申请日：2019-04-03

Applicant: Mujin, Inc.

Inventor： Jose Jeronimo Moreira Rodrigues ,  Xutao Ye ,  Jinze Yu ,  Rosen Diankov

IPC: B25J9/16 ,  B65G1/137 ,  G06K9/62 ,  G06K9/00

Abstract: A robotic system includes: a control unit configured to: receive an object set including one or more object entries, wherein: the object entries correspond to source objects of an object source, each of the object entries are described by one or more object entry properties; receive sensor information representing one or more detectable object properties for detectable source objects of an object source; calculate an object match probability between the detectable source objects and the object entries based on a property correlation between the detectable object properties of the detectable source objects and the object entry properties of the object entries; generate an object identity approximation for each of the detectable source objects based on a comparison between the object match probability for each of the detectable source objects corresponding to a particular instance of the object entries; select a target object from the detectable source objects; generate an object handling strategy, for implementation by an object handling unit, to transfer the target object from the object source based on the object entry properties of the object entries corresponding to the object identity approximation; update the object set to indicate that the target object corresponding to a specific instance of the object entries has been removed from the object source; and a storage unit 204, coupled to the control unit, configured to store the object set.

公开(公告)号：US10521930B1

公开(公告)日：2019-12-31

申请号：US16506695

申请日：2019-07-09

Applicant: MUJIN, INC.

Inventor： Russell Islam ,  Xutao Ye

IPC: G06T7/80 ,  H04N13/246 ,  G06T7/73

Abstract: A system and method for performing automatic camera calibration is presented. The system communicates with a first camera and a second camera, wherein a transparent platform is disposed between the two cameras. When a 3D calibration pattern is disposed on the platform, the system receives a first set of calibration images from the first camera, and a second set of calibration images from the second camera. The system determines, based on the first set of calibration images, a first set of coordinates for corners of the polyhedron. The system further determines, based on the second set of calibration images, a second set of coordinates for the corners. The system determines, based on the coordinates, a spatial relationship between the first camera and the second camera. The system further uses a description of the spatial relationship to generate a 3D model of an object other than the 3D calibration pattern.

公开(公告)号：US12138815B2

公开(公告)日：2024-11-12

申请号：US18331650

申请日：2023-06-08

Applicant: MUJIN, INC.

Inventor： Xutao Ye ,  Puttichai Lertkultanon ,  Rosen Nikolaev Diankov

IPC: G05B15/00 ,  B25J9/16 ,  B25J13/08 ,  B25J15/00 ,  B25J19/02 ,  B65G59/02 ,  G05B19/00 ,  G05B19/4155 ,  G06F18/2413 ,  G06T7/60 ,  G06T7/73 ,  G06V10/764 ,  G06V20/10 ,  H04N23/54 ,  H04N23/695

Abstract: A system and method for motion planning is presented. The system is configured, when an object is or has been in a camera field of view of a camera, to receive first image information that is generated when the camera has a first camera pose. The system is further configured to determine, based on the first image information, a first estimate of the object structure, and to identify, based on the first estimate of the object structure or based on the first image information, an object corner. The system is further configured to cause an end effector apparatus to move the camera to a second camera pose, and to receive second image information for representing the object's structure. The system is configured to determine a second estimate of the object's structure based on the second image information, and to generate a motion plan based on at least the second estimate.

公开(公告)号：US11958202B2

公开(公告)日：2024-04-16

申请号：US17400361

申请日：2021-08-12

Applicant: MUJIN, INC.

Inventor： Xutao Ye ,  Kazuto Murase

IPC: B25J9/16 ,  B25J13/08 ,  G06T7/50 ,  G06T7/73 ,  H04N23/695

CPC classification number: B25J9/1697 ,  B25J9/1666 ,  B25J9/1679 ,  B25J13/08 ,  G06T7/50 ,  G06T7/73 ,  H04N23/695 ,  G06T2207/10028 ,  G06T2207/30241

Abstract: A system and method for performing object detection are presented. The system receives spatial structure information associated with an object which is or has been in a camera field of view of a spatial structure sensing camera. The spatial structure information is generated by the spatial structure sensing camera, and includes depth information for an environment in the camera field of view. The system determines a container pose based on the spatial structure information, wherein the container pose is for describing at least one of an orientation for the container or a depth value for at least a portion of the container. The system further determines an object pose based on the container pose, wherein the object pose is for describing at least one of an orientation for the object or a depth value for at least a portion of the object.