公开(公告)号：US11501415B2

公开(公告)日：2022-11-15

申请号：US17080714

申请日：2020-10-26

申请人： Zili Yi ,  Qiang Tang ,  Shekoofeh Azizi ,  Daesik Jang ,  Zhan Xu

发明人： Zili Yi ,  Qiang Tang ,  Shekoofeh Azizi ,  Daesik Jang ,  Zhan Xu

IPC分类号： G06K9/00 ,  G06T5/00 ,  G06T3/40 ,  G06N3/08 ,  G06K9/62

摘要： Methods and systems for high-resolution image inpainting are disclosed. An original high-resolution image to be inpainted is obtained, as well as an inpainting mask indicating an inside-mask area to be inpainted. The original high-resolution image is down-sampled to obtain a low-resolution image to be inpainted. Using a trained inpainting generator, a low-resolution inpainted image and a set of attention scores are generated from the low-resolution image. The attention scores represent the similarity between inside-mask regions and outside-mask regions. A high-frequency residual image is computed from the original high-resolution image. An aggregated high-frequency residual image is generated using the attention scores, including high-frequency residual information for the inside-mask area. A high-resolution inpainted image is outputted by combining the aggregated high-frequency residual image and a low-frequency inpainted image generated from the low-resolution inpainted image.

公开(公告)号：US11915383B2

公开(公告)日：2024-02-27

申请号：US17367524

申请日：2021-07-05

申请人： Zili Yi ,  Qiang Tang ,  Vishnu Sanjay Ramiya Srinivasan ,  Zhan Xu

发明人： Zili Yi ,  Qiang Tang ,  Vishnu Sanjay Ramiya Srinivasan ,  Zhan Xu

IPC分类号： G06T3/00 ,  G06T3/40 ,  G06V40/16 ,  G06T11/60

CPC分类号： G06T3/0093 ,  G06T3/40 ,  G06T3/4046 ,  G06T3/4069 ,  G06T3/4076 ,  G06T11/60 ,  G06V40/162

摘要： Methods and systems for high-resolution image manipulation are disclosed. An original high-resolution image to be manipulated is obtained, as well as a driving signal indicating a manipulation result. The original high-resolution image is down-sampled to obtain a low-resolution image to be manipulated. Using a trained manipulation generator, a low-resolution manipulated image and a motion field are generated from the low-resolution image. The motion field represent pixel displacements of the low-resolution image to obtain the manipulation indicated by the driving signal. A high-frequency residual image is computed from the original high-resolution image. A high-frequency manipulated residual image is generated using the motion field. A high-resolution manipulated image is outputted by combining the high-frequency manipulated residual image and a low-frequency manipulated image generated from the low-resolution manipulated image by up-sampling.

公开(公告)号：US11676390B2

公开(公告)日：2023-06-13

申请号：US17079084

申请日：2020-10-23

申请人： Qiang Tang ,  Zili Yi ,  Zhan Xu

发明人： Qiang Tang ,  Zili Yi ,  Zhan Xu

IPC分类号： G06V20/52 ,  G06T7/11 ,  G06V40/16

CPC分类号： G06V20/53 ,  G06T7/11 ,  G06V40/162 ,  G06V40/172 ,  G06T2207/20084 ,  G06T2207/30242 ,  G06V2201/07

摘要： Methods and systems for fully-automatic image processing to detect and remove unwanted people from a digital image of a photograph. The system includes the following modules: 1) Deep neural network (DNN)-based module for object segmentation and head pose estimation; 2) classification (or grouping) of wanted versus unwanted people based on information collected in the first module; 3) image inpainting of the unwanted people in the digital image. The classification module can be rules-based in an example. In an example, the DNN-based module generates, from the digital image: 1. A list of object category labels, 2. A list of object scores, 3. A list of binary masks, 4. A list of object bounding boxes, 5. A list of crowd instances, 6. A list of human head bounding boxes, and 7. A list of head poses (e.g., yaws, pitches, and rolls).