公开(公告)号：US12093704B2

公开(公告)日：2024-09-17

申请号：US17228087

申请日：2021-04-12

申请人： Taslim Arefin Khan ,  Szu Wen Fan ,  Changqing Zou ,  Jianpeng Xu ,  Wei Li

发明人： Taslim Arefin Khan ,  Szu Wen Fan ,  Changqing Zou ,  Jianpeng Xu ,  Wei Li

IPC分类号： G06F9/451 ,  G06F3/04815 ,  G06F3/14 ,  G06T19/00 ,  G06F16/90

CPC分类号： G06F9/451 ,  G06F3/04815 ,  G06F3/1423 ,  G06T19/006 ,  G06F16/90

摘要： Devices, methods, systems, and media are described for providing an extended screen distributed user interface in an augmented reality environment. GUI layout information for laying out a conventional 2D GUI is processed in order to generate an extended screen DUI for display partially on a 2D display device and partially on one or more virtual screens of an AR environment viewed using an AR display device, such as a head mounted display. GUI elements are laid out in the DUI based on a primary modality of the GUI element (input or output), and/or based on spatial dependencies between GUI elements encoded in the GUI layout information. Methods for switching focus between two software application instances displayed in the DUI are also disclosed.

公开(公告)号：US11640700B2

公开(公告)日：2023-05-02

申请号：US17187663

申请日：2021-02-26

申请人： Changqing Zou ,  Yumna Wassaf Akhtar ,  Szu Wen Fan ,  Jianpeng Xu ,  Wei Li

发明人： Changqing Zou ,  Yumna Wassaf Akhtar ,  Szu Wen Fan ,  Jianpeng Xu ,  Wei Li

IPC分类号： G06V10/25 ,  G06F3/01 ,  G06T19/00 ,  G06V20/20 ,  G06V40/20

摘要： Methods and systems for rendering a virtual overlay in an extended reality (XR) environment are described. Image data is received, representing a current field of view (FOV) in the XR environment. A spatial boundary is defined in the current FOV, based on user input. An image label representing a region of interest (ROI) within the defined spatial boundary, and one or more object labels representing one or more objects within the defined spatial boundary are generated. At least one relevant virtual object is identified. The virtual object is relevant to a semantic meaning of the ROI based on the image label and/or the one or more object labels. Identification of the at least one relevant virtual object is outputted, to be renderable in the XR environment.

公开(公告)号：US11328172B2

公开(公告)日：2022-05-10

申请号：US17001361

申请日：2020-08-24

申请人： Changqing Zou ,  Wei Li

发明人： Changqing Zou ,  Wei Li

IPC分类号： G06K9/62 ,  G06F16/56 ,  G06F16/583 ,  G06N3/04 ,  G06T7/11 ,  G06V10/426 ,  G06V10/44

摘要： A sketch-based image retrieval method, device and system, to improve accuracy of image searching from a scene sketch image. For example, the image retrieval method, device and system can be used to retrieve a target scene image from a collection of stored images in a storage (i.e., an image collection). The image retrieval method includes: segmenting the scene sketch image using an image segmentation module into semantic object-level instances, and fine-grained features are obtained for each object instance, generating an attribute graph which integrates the fine-grained features for each semantic object instance detected from the query scene sketch image, generating a feature graph by using a graph encoder module from the attribute graph, and computing a similarity or distance between the feature graphs of the query scene sketch image and the scene images in the image collection by a graph matching module and the most similar scene images are returned.

公开(公告)号：US11327630B1

公开(公告)日：2022-05-10

申请号：US17167764

申请日：2021-02-04

申请人： Taslim Arefin Khan ,  Szu Wen Fan ,  Changqing Zou ,  Wei Li

发明人： Taslim Arefin Khan ,  Szu Wen Fan ,  Changqing Zou ,  Wei Li

IPC分类号： G06F3/0481 ,  G06T19/00 ,  G06F3/01 ,  G02B27/01 ,  H04B1/3827 ,  G06F3/04815

摘要： Devices, methods, systems, and media are described for selecting virtual objects for user interaction in an extended reality environment. Distant virtual objects are brought closer to the user within a virtual 3D space to situate the selected virtual object in virtual proximity to the user's hand for direct manipulation. A virtual object is selected by the user based on movements of the user's hand and/or head that are correlated or associated with an intent to select a specific virtual object within the virtual 3D space. As the user's hand moves in a way that is consistent with this intent, the virtual object is brought closer to the user's hand within the virtual 3D space. To predict the user's intent, hand and head trajectory data may be compared to a library of kinematic trajectory templates to identify a best-matched trajectory template.

公开(公告)号：US11928759B2

公开(公告)日：2024-03-12

申请号：US17724452

申请日：2022-04-19

申请人： Changqing Zou ,  Mingxue Wang ,  Himanshu Arora

发明人： Changqing Zou ,  Mingxue Wang ,  Himanshu Arora

IPC分类号： G06T11/20 ,  G06F3/04883 ,  G06T11/00

CPC分类号： G06T11/203 ,  G06F3/04883 ,  G06T11/001

摘要： The present disclosure describes methods and devices for generating a vector line drawing. A vector line drawing network may include a machine learning-based model that is trained to convert a raster image to a vector line drawing directly. The vector line drawing network may be trained end-to-end, using supervised learning, where only raster images are used as training data. A vector line drawing is generated stroke by stroke, over a series of time steps. In each time step, a dynamic drawing window is moved and scaled across the input raster image to sample a patch of the raster image, and a drawing stroke is predicted to draw a stroke in a corresponding patch in the canvas for the vector line drawing. The image patches are pasted in the canvas to assemble a final vector line drawing that corresponds to the input raster image.