公开(公告)号：US11288810B2

公开(公告)日：2022-03-29

申请号：US16888376

申请日：2020-05-29

Applicant: MUJIN, Inc.

Inventor： Rosen Diankov ,  Huan Liu ,  Xutao Ye ,  Jose Jeronimo Moreira Rodrigues ,  Yoshiki Kanemoto ,  Jinze Yu ,  Russell Islam

IPC: G06T7/13 ,  G06T7/73 ,  G06T7/50 ,  G06T7/62 ,  G06T19/20 ,  G06T7/11 ,  G06K9/00 ,  G06K9/32 ,  B25J9/16 ,  B65G61/00 ,  G05B19/4093 ,  G06Q10/08 ,  B65B43/46

Abstract: A method for operating a robotic system includes obtaining and processing first data representative of an object at a start location. An event may be detected while implementing an operation based on the image data. A gripper height that corresponds to the event may be determined. Accordingly, the method may include calculating an object height that represents a height estimate of the object.

公开(公告)号：US11176674B2

公开(公告)日：2021-11-16

申请号：US16667882

申请日：2019-10-29

Applicant: MUJIN, Inc.

Inventor： Rosen Nikolaev Diankov ,  Russell Islam ,  Xutao Ye

IPC: G06T7/11 ,  G06T7/13 ,  B25J9/16 ,  G06T7/73 ,  G06T7/50 ,  G06T7/62 ,  B65G61/00 ,  G05B19/4093 ,  G06Q10/08 ,  G06T19/20 ,  B65B43/46 ,  G06K9/00 ,  G06K9/32

Abstract: A system and method for operating a robotic system to register unrecognized objects is disclosed. The robotic system may use first image data representative of an unrecognized object located at a start location to derive an initial minimum viable region (MVR) and to implement operations for initially displacing the unrecognized object. The robotic system may analyze second image data representative of the unrecognized object after the initial displacement operations to detect a condition representative of an accuracy of the initial MVR. The robotic system may register the initial MVR or an adjustment thereof based on the detected condition.

公开(公告)号：US11080876B2

公开(公告)日：2021-08-03

申请号：US16668492

申请日：2019-10-30

Applicant: MUJIN, INC.

Inventor： Russell Islam ,  Xutao Ye

IPC: G06T7/55 ,  B25J9/16 ,  G06T7/70 ,  H04N5/247 ,  G06T7/521

Abstract: A method and system for processing camera images is presented. The system receives a first depth map generated based on information sensed by a first type of depth-sensing camera, and receives a second depth map generated based on information sensed by a second type of depth-sensing camera. The first depth map includes a first set of pixels that indicate a first set of respective depth values. The second depth map includes a second set of pixels that indicate a second set of respective depth values. The system identifies a third set of pixels of the first depth map that correspond to the second set of pixels of the second depth map, identifies one or more empty pixels from the third set of pixels, and updates the first depth map by assigning to each empty pixel a respective depth value based on the second depth map.

公开(公告)号：US11006039B1

公开(公告)日：2021-05-11

申请号：US16789570

申请日：2020-02-13

Applicant: MUJIN, INC.

Inventor： Jinze Yu ,  Jose Jeronimo Moreira Rodrigues ,  Xutao Ye

IPC: H04N5/232 ,  H04N5/217 ,  B25J9/16

Abstract: A system and method for determining occlusion are presented. The system receives camera data generated by at least one camera, which includes a first camera having a first camera field of view. The camera data is generated when a stack having a plurality of objects is in the first camera field of view, and describes a stack structure formed from at least an object structure for a first object of the plurality of objects. The system identifies a target feature of or disposed on the object structure, and determines a 2D region that is co-planar with and surrounds the target feature. The system determines a 3D region defined by connecting a location of the first camera and the 2D region. The system determines, based on the camera data and the 3D region, a size of an occluding region, and determines a value of an object recognition confidence parameter.

公开(公告)号：US10970866B2

公开(公告)日：2021-04-06

申请号：US16739184

申请日：2020-01-10

Applicant: MUJIN, Inc.

Inventor： Rosen Diankov ,  Xutao Ye ,  Ziyan Zhou

IPC: G06T7/55 ,  B65G67/24 ,  B65G67/04 ,  B65G67/02 ,  B25J9/00 ,  B25J9/16 ,  B65G59/02

Abstract: A control apparatus includes a first information acquiring section that acquires three-dimensional information of a first region of surfaces of a plurality of objects, the information being obtained by imaging or scanning the plurality of objects from a first location; a second information acquiring section that acquires three-dimensional information of a second region of surfaces of the plurality of objects, the information being obtained by imaging or scanning the plurality of objects from a second location; and a combining section that generates information indicating three-dimensional shapes of at least a portion of the surfaces of the plurality of objects, based on the three-dimensional information of the first region acquired by the first information acquiring section and the three-dimensional information of the second region acquired by the second information acquiring section.

公开(公告)号：US10906184B2

公开(公告)日：2021-02-02

申请号：US16864071

申请日：2020-04-30

Applicant: MUJIN, INC.

Inventor： Russell Islam ,  Xutao Ye ,  Rosen Diankov

IPC: B25J9/16 ,  G06T7/80

Abstract: A computing system and a method for calibration verification is presented. The computing system is configured to perform a first calibration operation, and to control a robot arm to move a verification symbol to a reference location. The robot control system further receives, from a camera, a reference image of the verification symbol, and determines a reference image coordinate for the verification symbol. The robot control system further controls the robot arm to move the verification symbol to the reference location again during an idle period, receives an additional image of the verification symbol, and determines a verification image coordinate. The robot control system determines a deviation parameter value based the reference image coordinate and the verification image coordinate, and whether the deviation parameter value exceeds a defined threshold, and performs a second calibration operation if the threshold is exceeded.

公开(公告)号：US20200294244A1

公开(公告)日：2020-09-17

申请号：US16888376

申请日：2020-05-29

Applicant: MUJIN, Inc.

Inventor： Rosen Nikolaev Diankov ,  Huan Liu ,  Xutao Ye ,  Jose Jeronimo Moreira Rodrigues ,  Yoshiki Kanemoto ,  Jinze Yu ,  Russell Islam

IPC: G06T7/13 ,  B25J9/16 ,  G06T7/73 ,  G06T7/50 ,  G06T7/62 ,  B65G61/00 ,  G05B19/4093 ,  G06Q10/08 ,  G06T19/20 ,  G06T7/11 ,  B65B43/46 ,  G06K9/00 ,  G06K9/32

Abstract: A method for operating a robotic system includes obtaining and processing first data representative of an object at a start location. An event may be detected while implementing an operation based on the image data. A gripper height that corresponds to the event may be determined. Accordingly, the method may include calculating an object height that represents a height estimate of the object.

公开(公告)号：US10562189B1

公开(公告)日：2020-02-18

申请号：US16443757

申请日：2019-06-17

Applicant: MUJIN, Inc.

Inventor： Rosen Diankov ,  Huan Liu ,  Xutao Ye ,  Jose Jeronimo Moreira Rodrigues ,  Yoshiki Kanemoto ,  Jinze Yu ,  Russell Islam

IPC: B25J9/16 ,  G06T7/00 ,  B25J13/08 ,  G06T7/50 ,  G06T7/13 ,  G06K9/62

Abstract: A method of operating a package handling system includes: receiving first image data representative of a first surface image of a package surface; identifying a pair of intersecting edges for the package surface based on the first image; determining a minimum viable region based on the pair of edges, the minimum viable region for gripping and lifting the package; receiving second image data representative of the package after the lift; and creating registration data based on the third image data.

公开(公告)号：US12219269B2

公开(公告)日：2025-02-04

申请号：US18591980

申请日：2024-02-29

Applicant: MUJIN, INC.

Inventor： Jinze Yu ,  Jose Jeronimo Moreira Rodrigues ,  Xutao Ye

IPC: H04N23/80 ,  B25J9/16 ,  H04N23/698 ,  H04N23/81

Abstract: A system and method for determining occlusion are presented. The system receives camera data generated by at least one camera, which includes a first camera having a first camera field of view. The camera data is generated when a stack having a plurality of objects is in the first camera field of view, and describes a stack structure formed from at least an object structure for a first object of the plurality of objects. The system identifies a target feature of or disposed on the object structure, and determines a 2D region that is co-planar with and surrounds the target feature. The system determines a 3D region defined by connecting a location of the first camera and the 2D region. The system determines, based on the camera data and the 3D region, a size of an occluding region, and determines a value of an object recognition confidence parameter.

公开(公告)号：US20240189982A1

公开(公告)日：2024-06-13

申请号：US18532868

申请日：2023-12-07

Applicant: MUJIN, Inc.

Inventor： Rosen Nikolaev Diankov ,  Puttichai Lertkultanon ,  Shintaro Matsuoka ,  Yoshiki Kanemoto ,  Jose Jeronimo Moreira Rodrigues ,  Lei Lei ,  Yixuan Zhang ,  Xutao Ye ,  Yufan Du ,  Mingjian Liang ,  Lingping Gao ,  Xinhao Wen ,  Xu Chen

IPC: B25J9/00 ,  B25J5/00 ,  B65G67/08 ,  B65G67/24

CPC classification number: B25J9/0093 ,  B25J5/007 ,  B65G67/08 ,  B65G67/24

Abstract: A robotic system may include a chassis as well as a first leg and a second leg operatively coupled to the chassis. The first leg and the second leg may be configured to move in a vertical direction to move the chassis in a vertical translational degree of freedom. The robotic system may include a proximal conveyor, a first segment including a first segment conveyor extending along a length of the first segment, and a gripper including a distal conveyor extending along a length of the gripper. A first joint between the proximal conveyor and the first segment may be configured to provide a first rotational degree of freedom between the first segment and the proximal conveyor. A second joint between the gripper and the first segment may be configured to provide a second rotational degree of freedom between the first segment and the gripper.