公开(公告)号：US20240087257A1

公开(公告)日：2024-03-14

申请号：US18507293

申请日：2023-11-13

申请人： Fannie Liu ,  Rajan Vaish

发明人： Fannie Liu ,  Rajan Vaish

IPC分类号： G06T19/00 ,  G06F3/04842 ,  G06F3/04845 ,  G08B5/22 ,  H04N5/91

CPC分类号： G06T19/006 ,  G06F3/04842 ,  G06F3/04845 ,  G08B5/22 ,  H04N5/91 ,  G06T2200/24 ,  G06T2219/024

摘要： Systems and methods enable users to engage in meaningful, authentic, online interactions by extracting objects (virtual or real) from an image or video and transferring the extracted objects into a real-world environment of another user in three-dimensional augmented reality. An object to be sent from the first user to the second user is generated using a drawing application, extracted from an image or a video, or an AR object is captured. A video is recorded showing the object responding to an action of the first user and metadata is generated relating to the response of the object to the action of the first user. The video and metadata are sent to the second user. Upon receipt, the video is watched by the second user and the object is selected for presentation in a display environment of the second user. The state of the object is preserved by the metadata.

公开(公告)号：US20240077983A1

公开(公告)日：2024-03-07

申请号：US17901589

申请日：2022-09-01

申请人： Lei Zhang ,  Youjean Cho ,  Daekun Kim ,  Ava Robinson ,  Yu Jiang Tham ,  Rajan Vaish ,  Andrés Monroy-Hernández

发明人： Lei Zhang ,  Youjean Cho ,  Daekun Kim ,  Ava Robinson ,  Yu Jiang Tham ,  Rajan Vaish ,  Andrés Monroy-Hernández

IPC分类号： G06F3/04815 ,  G06F3/01 ,  G06F3/04842 ,  G06F3/16 ,  G06F9/54 ,  G06T7/70 ,  G06T19/00 ,  G06V20/20

CPC分类号： G06F3/04815 ,  G06F3/011 ,  G06F3/04842 ,  G06F3/167 ,  G06F9/541 ,  G06T7/70 ,  G06T19/006 ,  G06V20/20 ,  G06T2200/24 ,  G06T2207/30244

摘要： Recording tools for creating interactive AR experiences. An interaction recording application enables a user with little or no programming skills to perform and record user behaviors that are associated with reactions between story elements such as virtual objects and connected IoT devices. The user behaviors include a range of actions, such as speaking a trigger word and apparently touching a virtual object. The corresponding reactions include starting to record a subsequent scene and executing actions between story elements. The trigger recording interface is presented on the display as an overlay relative to the physical environment.

公开(公告)号：US20240077935A1

公开(公告)日：2024-03-07

申请号：US17901630

申请日：2022-09-01

申请人： Youjean Cho ,  Lei Zhang ,  Daekun Kim ,  Ava Robinson ,  Yu Jiang Tham ,  Rajan Vaish ,  Andrés Monroy-Hernández

发明人： Youjean Cho ,  Lei Zhang ,  Daekun Kim ,  Ava Robinson ,  Yu Jiang Tham ,  Rajan Vaish ,  Andrés Monroy-Hernández

IPC分类号： G06F3/01 ,  G06F3/033 ,  G06V10/20 ,  G06V40/20 ,  G16Y20/40 ,  H04N13/111

CPC分类号： G06F3/011 ,  G06F3/017 ,  G06F3/0334 ,  G06V10/255 ,  G06V40/20 ,  G16Y20/40 ,  H04N13/111

摘要： A virtual interface application presented in augmented reality (AR) is described for controlling Internet of Things (IoT) products. The virtual interface application enables a user of a portable electronic device to activate a virtual control interface overlay on a display, receive a selection from the user using her hands or feet, and send a control signal to a nearby IoT product which executes an action in accordance with the selection. The virtual control interface is presented on the display as a virtual object in an AR environment. The virtual interface application includes a foot tracking tool for detecting an intersection between the foot location (in the physical environment) and the virtual surface position (in the AR environment). When an intersection is detected, the virtual interface application sends a control signal with instructions to the IoT product.

公开(公告)号：US20240077933A1

公开(公告)日：2024-03-07

申请号：US17901483

申请日：2022-09-01

申请人： Daekun Kim ,  Lei Zhang ,  Youjean Cho ,  Ava Robinson ,  Yu Jiang Tham ,  Rajan Vaish ,  Andrés Monroy-Hernández

发明人： Daekun Kim ,  Lei Zhang ,  Youjean Cho ,  Ava Robinson ,  Yu Jiang Tham ,  Rajan Vaish ,  Andrés Monroy-Hernández

IPC分类号： G06F3/01 ,  G06T7/70 ,  G06T19/00 ,  G06V40/20 ,  H04W4/06

CPC分类号： G06F3/011 ,  G06T7/70 ,  G06T19/006 ,  G06V40/23 ,  H04W4/06 ,  G06T2219/024

摘要： A method for detecting full-body gestures by a mobile device includes a host mobile device detecting the tracked body of a co-located participant in a multi-party session. When the participant's tracked body provides a full-body gesture, the host's mobile device recognizes that there is a tracked body providing a full-body gesture. The host mobile device iterates through the list of participants in the multi-party session and finds the closest participant mobile device with respect to the screen-space position of the head of the gesturing participant. The host mobile device then obtains the user ID of the closest participant mobile device and broadcasts the recognized full-body gesture event to all co-located participants in the multi-party session, along with the obtained user ID. Each participant's mobile device may then handle the gesture event as appropriate for the multi-party session. For example, a character or costume may be assigned to a gesturing participant.

公开(公告)号：US20240070299A1

公开(公告)日：2024-02-29

申请号：US17900792

申请日：2022-08-31

申请人： Youjean Cho ,  Chen Ji ,  Fannie Liu ,  Andrés Monroy-Hernández ,  Tsung-Yu Tsai ,  Rajan Vaish

发明人： Youjean Cho ,  Chen Ji ,  Fannie Liu ,  Andrés Monroy-Hernández ,  Tsung-Yu Tsai ,  Rajan Vaish

IPC分类号： G06F21/62 ,  G06T19/20

CPC分类号： G06F21/6209 ,  G06T19/20 ,  G06F2221/2113 ,  G06F2221/2137 ,  G06T2219/024

摘要： A collaborative session (e.g., a virtual time capsule) in which access to a collaborative object and added virtual content is selectively provided to participants/users. In one example of the collaborative session, a processor provides users with access to a collaborative object using respective physically remote devices, and associates virtual content received from the users with the collaborative object during a collaboration period. The processor maintains a timer including a countdown indicative of when the collaboration period ends for associating virtual content with the collaborative object. The processor provides the users with access to the collaborative object with associated virtual content at the end of the collaboration period.

公开(公告)号：US20240070298A1

公开(公告)日：2024-02-29

申请号：US17900783

申请日：2022-08-31

申请人： Youjean Cho ,  Chen Ji ,  Fannie Liu ,  Andrés Monroy-Hernández ,  Tsung-Yu Tsai ,  Rajan Vaish

发明人： Youjean Cho ,  Chen Ji ,  Fannie Liu ,  Andrés Monroy-Hernández ,  Tsung-Yu Tsai ,  Rajan Vaish

IPC分类号： G06F21/62 ,  G06T19/20

CPC分类号： G06F21/6209 ,  G06T19/20 ,  G06F2221/2113 ,  G06F2221/2137 ,  G06T2219/024

摘要： Collaborative sessions in which access to added virtual content is selectively made available to participants/users. A participant (the host) creates a new session and invites participants to join. The invited participants receive an invitation to join the session. The session creator (i.e., the host) and other approved participants can access the contents of a session. The session identifies a new participant when they join the session, and concurrently notifies the other participants in the session that a new participant is waiting for permission to access the added virtual content. The host or approved participants can set up the new participant with permissions for accessing added virtual content.

公开(公告)号：US11875022B2

公开(公告)日：2024-01-16

申请号：US17460228

申请日：2021-08-29

申请人： Yu Jiang Tham ,  Rajan Vaish ,  Andrés Monroy-Hernández ,  Sven Kratz ,  Ana Maria Cardenas Gasca

发明人： Yu Jiang Tham ,  Rajan Vaish ,  Andrés Monroy-Hernández ,  Sven Kratz ,  Ana Maria Cardenas Gasca

IPC分类号： G06V20/20 ,  G06F3/0484 ,  H04N23/63 ,  G06F3/01 ,  G06F3/0482 ,  G16Y40/35

CPC分类号： G06F3/0484 ,  G06F3/017 ,  G06F3/0482 ,  G06V20/20 ,  H04N23/63 ,  G16Y40/35

摘要： Systems and methods for interfacing augmented reality (AR) camera devices to Internet of Things (IoT) devices include pairing an AR camera device to an IoT device to establish which actions recognized by the AR camera device may control the IoT device, receiving an action identifier representing a gesture or position of a body part that has been recognized by the AR camera device, and sending a command to the IoT device paired with the AR camera device to perform a requested action in response to the action identifier received from the AR camera device. Context information relating to the action identifier from the AR camera may be used to modify the command sent to the IoT device. The AR camera device may be paired with an IoT device pointed at by the AR camera device, selected through a user interface of the AR camera device, or determined from context information.

公开(公告)号：US20220414989A1

公开(公告)日：2022-12-29

申请号：US17362431

申请日：2021-06-29

申请人： Rajan Vaish ,  Yu Jiang Tham ,  Brian Anthony Smith ,  Sven Kratz ,  Karen Stoizenberg ,  David Meisenholder

发明人： Rajan Vaish ,  Yu Jiang Tham ,  Brian Anthony Smith ,  Sven Kratz ,  Karen Stoizenberg ,  David Meisenholder

IPC分类号： G06T19/00 ,  G06T13/80 ,  G06T7/00 ,  H04L12/58

摘要： A virtual mailbox is established for a first user based on a virtual mailbox definition specified by the first user. A message directed to the first user is received from a first device associated with a second user. A location within a real-world environment of the first user corresponding to the virtual mailbox is identified. A marker associated with the virtual mailbox is detected within the real-world environment of the first user. Based on detecting the marker, the second device presents the message overlaid on the real-world environment at the location corresponding to the virtual mailbox.

公开(公告)号：US20240320353A1

公开(公告)日：2024-09-26

申请号：US18679847

申请日：2024-05-31

申请人： Youjean Cho ,  Chen Ji ,  Fannie Liu ,  Andrés Monroy-Hernández ,  Tsung-Yu Tsai ,  Rajan Vaish

发明人： Youjean Cho ,  Chen Ji ,  Fannie Liu ,  Andrés Monroy-Hernández ,  Tsung-Yu Tsai ,  Rajan Vaish

IPC分类号： G06F21/62 ,  G06T19/20

CPC分类号： G06F21/6209 ,  G06T19/20 ,  G06F2221/2137 ,  G06F2221/2151 ,  G06T2219/024

摘要： A collaborative session (e.g., a virtual time capsule) in which access to a collaborative object and added virtual content is selectively provided to participants/users. In one example of the collaborative session, a participant (the host) creates a new session and invites participants to join. The session creator (i.e., the host) and other approved participants can access the contents of a session (e.g., which may be recorded using an application such as lens cloud feature; available from Snap Inc. of Santa Monica, California). A timestamp is associated with each received virtual content, and the users are provided with a timelapse of the collaborative object as a function of the timestamps.

公开(公告)号：US20240079031A1

公开(公告)日：2024-03-07

申请号：US17901540

申请日：2022-09-01

申请人： Lei Zhang ,  Daekun Kim ,  Youjean Cho ,  Ava Robinson ,  Yu Jiang Tham ,  Rajan Vaish ,  Andrés Monroy-Hernández

发明人： Lei Zhang ,  Daekun Kim ,  Youjean Cho ,  Ava Robinson ,  Yu Jiang Tham ,  Rajan Vaish ,  Andrés Monroy-Hernández

IPC分类号： G11B27/031 ,  G06F3/04817 ,  G06F3/04842 ,  G06F3/16 ,  G06T19/00 ,  G06V20/20 ,  G10L15/08 ,  G10L15/22 ,  G11B27/34

CPC分类号： G11B27/031 ,  G06F3/04817 ,  G06F3/04842 ,  G06F3/167 ,  G06T19/006 ,  G06V20/20 ,  G10L15/08 ,  G10L15/22 ,  G11B27/34 ,  G06T2200/24 ,  G10L2015/088

摘要： Described are authoring tools for creating interactive AR experiences. The story-authoring application enables a user with little or no programming skills to create an interactive story that includes recording voice commands for advancing to the next scene, inserting and manipulating virtual objects in a mixed-reality environment, and recording a variety of interactions with connected IoT devices. The story creation interface is presented on the display as a virtual object in an AR environment.