公开(公告)号：US12282696B2

公开(公告)日：2025-04-22

申请号：US18723429

申请日：2022-12-18

Applicant: Yeda Research and Development Co. Ltd.

Inventor： Tali Dekel ,  Shai Bagon ,  Omer Bar Tal ,  Narek Tumanyan

IPC: G06F3/14 ,  G06T7/11 ,  G06V10/54 ,  G06V10/56 ,  G06V10/74 ,  G06V20/70

Abstract: Using a pre-trained and fixed Vision Transformer (ViT) model as an external semantic prior, a generator is trained given only a single structure/appearance image pair as input. Given two input images, a source structure image and a target appearance image, a new image is generated by the generator in which the structure of the source image is preserved, while the visual appearance of the target image is transferred in a semantically aware manner, so that objects in the structure image are “painted” with the visual appearance of semantically related objects in the appearance image. A self-supervised, pre-trained ViT model, such as a DINO-VIT model, is leveraged as an external semantic prior, allowing for training of the generator only on a single input image pair, without any additional information (e.g., segmentation/correspondences), and without adversarial training. The method may generate high quality results in high resolution (e.g., HD).