公开(公告)号：US10964025B2

公开(公告)日：2021-03-30

申请号：US16739115

申请日：2020-01-10

Applicant: Qingdao University of Technology

Inventor： Chengjun Chen ,  Chunlin Zhang ,  Dongnian Li ,  Jun Hong

IPC: G06T7/11 ,  G06T19/20 ,  H04N7/18

Abstract: The present invention relates to an assembly monitoring method based on deep learning, comprising steps of: creating a training set for a physical assembly body, the training set comprising a depth image set Di and a label image set Li of a 3D assembly body at multiple monitoring angles, wherein i represents an assembly step, the depth image set Di in the ith step corresponds to the label image set Li in the ith step, and in label images in the label image set Li, different parts of the 3D assembly body are rendered by different colors; training a deep learning network model by the training set; and obtaining, by the depth camera, a physical assembly body depth image C in a physical assembly scene, inputting the physical assembly body depth image C into the deep learning network model, and outputting a pixel segmentation image of the physical assembly body, in which different parts are represented by pixel colors to identify all the parts of the physical assembly body. In the present invention, parts in the assembly body can be identified, and the assembly steps, as well as the occurrence of assembly errors and the type of errors, can be monitored for the parts.

公开(公告)号：US20200273177A1

公开(公告)日：2020-08-27

申请号：US16739115

申请日：2020-01-10

Applicant: Qingdao University of Technology

Inventor： Chengjun Chen ,  Chunlin Zhang ,  Dongnian Li ,  Jun Hong

IPC: G06T7/11 ,  G06T19/20 ,  H04N7/18

Abstract: The present invention relates to an assembly monitoring method based on deep learning, comprising steps of: creating a training set for a physical assembly body, the training set comprising a depth image set Di and a label image set Li of a 3D assembly body at multiple monitoring angles, wherein i represents an assembly step, the depth image set Di in the ith step corresponds to the label image set Li in the ith step, and in label images in the label image set Li, different parts of the 3D assembly body are rendered by different colors; training a deep learning network model by the training set; and obtaining, by the depth camera, a physical assembly body depth image C in a physical assembly scene, inputting the physical assembly body depth image C into the deep learning network model, and outputting a pixel segmentation image of the physical assembly body, in which different parts are represented by pixel colors to identify all the parts of the physical assembly body. In the present invention, parts in the assembly body can be identified, and the assembly steps, as well as the occurrence of assembly errors and the type of errors, can be monitored for the parts.