公开(公告)号：US11062457B2

公开(公告)日：2021-07-13

申请号：US16667822

申请日：2019-10-29

Applicant: MUJIN, Inc.

Inventor： Jinze Yu ,  Jose Jeronimo Moreira Rodrigues ,  Rosen Nikolaev Diankov ,  Xutao Ye

IPC: G06K9/00 ,  G06T7/13 ,  B25J9/16 ,  G06T7/73 ,  G06T7/50 ,  G06T7/62 ,  B65G61/00 ,  G05B19/4093 ,  G06Q10/08 ,  G06T19/20 ,  G06T7/11 ,  B65B43/46 ,  G06K9/32

Abstract: The present disclosure relates to methods and systems for generating a verified minimum viable range (MVR) of an object. An exposed outer corner and exposed edges of an object may be identified by processing one or more image data. An initial MVR may be generated by identifying opposing parallel edges opposing the exposed edges. The initial MVR may be adjusted, and the adjusted result may be tested to generate a verified MVR.

公开(公告)号：US11034025B2

公开(公告)日：2021-06-15

申请号：US16736667

申请日：2020-01-07

Applicant: MUJIN, Inc.

Inventor： Rosen Diankov ,  Huan Liu ,  Xutao Ye ,  Jose Jeronimo Moreira Rodrigues ,  Yoshiki Kanemoto ,  Jinze Yu ,  Russell Islam

IPC: B25J9/16 ,  G06T7/13 ,  G06T7/50 ,  G06K9/62 ,  B25J13/08 ,  G06T7/00 ,  G06K9/46 ,  G06K9/00

Abstract: A method of operating a robotic system includes: receiving first image data representative of a package surface; identifying a pair of edges based on the first image data; determining a minimum viable region based on the pair of edges; receiving second image data representative of the package after the lift; and creating registration data based on the first and second image data.

公开(公告)号：US20200130961A1

公开(公告)日：2020-04-30

申请号：US16539790

申请日：2019-08-13

Applicant: MUJIN, Inc.

Inventor： Rosen Nikolaev Diankov ,  Huan Liu ,  Xutao Ye ,  Jose Jeronimo Moreira Rodrigues ,  Yoshiki Kanemoto ,  Jinze Yu ,  Russell Islam

IPC: B65G61/00 ,  B25J9/16 ,  G05B19/4093 ,  G06Q10/08

Abstract: A method for operating a robotic system includes obtaining and processing first data representative of an object at a start location; obtaining additional data as the object is transferred to a task location, wherein the additional data includes information regarding one or more edges and/or one or more surfaces separate from portions of the object captured in the first data; and creating registration data representative of the object based on the additional data.

公开(公告)号：US10576636B1

公开(公告)日：2020-03-03

申请号：US16383389

申请日：2019-04-12

Applicant: MUJIN, INC.

Inventor： Russell Islam ,  Xutao Ye ,  Rosen Diankov

IPC: B25J13/08 ,  B25J9/16 ,  G06T7/80

Abstract: A robot control system and a method for updating camera calibration is presented. The method comprises the robot control system performing a first camera calibration to determine camera calibration information, and outputting a first movement command based on the camera calibration information for a robot operation. The method further comprises outputting, after the first camera calibration, a second movement command to move a calibration pattern within a camera field of view, receiving one or more calibration images, and adding the one or more calibration images to a captured image set. The method further comprises performing a second camera calibration based on calibration images in the captured image set to determine updated camera calibration information, determining whether a deviation between the camera calibration information and the updated camera calibration information exceeds a defined threshold, and outputting a notification signal if the deviation exceeds the defined threshold.

公开(公告)号：US10562188B1

公开(公告)日：2020-02-18

申请号：US16443743

申请日：2019-06-17

Applicant: MUJIN, Inc.

Inventor： Rosen Diankov ,  Huan Liu ,  Xutao Ye ,  Jose Jeronimo Moreira Rodrigues ,  Yoshiki Kanemoto ,  Jinze Yu ,  Russell Islam

IPC: B25J9/16 ,  G06T7/00 ,  B25J13/08 ,  G06T7/50 ,  G06T7/13 ,  G06K9/62

Abstract: A method of operating a package handling system includes: receiving first image data representative of a first surface image of a package surface; identifying a pair of intersecting edges for the package surface based on the first image; determining a minimum viable region based on the pair of edges, the minimum viable region for gripping and lifting the package; receiving second image data representative of the package after the lift; and creating registration data based on the third image data.

公开(公告)号：US10510155B1

公开(公告)日：2019-12-17

申请号：US16438084

申请日：2019-06-11

Applicant: MUJIN, INC.

Inventor： Russell Islam ,  Xutao Ye

IPC: G06T7/55 ,  G06T7/70 ,  B25J9/16 ,  H04N5/247 ,  G06T7/521

Abstract: A method and system for processing camera images is presented. The system receives a first depth map generated based on information sensed by a first type of depth-sensing camera, and receives a second depth map generated based on information sensed by a second type of depth-sensing camera. The first depth map includes a first set of pixels that indicate a first set of respective depth values. The second depth map includes a second set of pixels that indicate a second set of respective depth values. The system identifies a third set of pixels of the first depth map that correspond to the second set of pixels of the second depth map, identifies one or more empty pixels from the third set of pixels, and updates the first depth map by assigning to each empty pixel a respective depth value based on the second depth map.

公开(公告)号：US10373336B1

公开(公告)日：2019-08-06

申请号：US16362471

申请日：2019-03-22

Applicant: MUJIN, INC.

Inventor： Russell Islam ,  Xutao Ye ,  Rosen Diankov

IPC: G06T7/80 ,  B25J9/16 ,  G06T7/73 ,  G06T5/00 ,  G06T7/60 ,  G06T7/50

Abstract: A robot control system and a method for automatic camera calibration is presented. The robot control system includes a control circuit configured to control a robot arm to move a calibration pattern to at least one location within a camera field of view, and to receive a calibration image from a camera. The control circuit determines a first estimate of a first intrinsic camera parameter based on the calibration image. After the first estimate of the first intrinsic camera parameter is determined, the control circuit determines a first estimate of a second intrinsic camera parameter based on the first estimate of the first intrinsic camera parameter. These estimates are used to determine an estimate of a transformation function that describes a relationship between a camera coordinate system and a world coordinate system. The control circuit controls placement of the robot arm based on the estimate of the transformation function.