公开(公告)号：US12073011B2

公开(公告)日：2024-08-27

申请号：US17901630

申请日：2022-09-01

申请人： Youjean Cho ,  Lei Zhang ,  Daekun Kim ,  Ava Robinson ,  Yu Jiang Tham ,  Rajan Vaish ,  Andrés Monroy-Hernández

发明人： Youjean Cho ,  Lei Zhang ,  Daekun Kim ,  Ava Robinson ,  Yu Jiang Tham ,  Rajan Vaish ,  Andrés Monroy-Hernández

IPC分类号： G06F3/01 ,  G06F3/033 ,  G06V10/20 ,  G06V40/20 ,  G16Y20/40 ,  H04N13/111

CPC分类号： G06F3/011 ,  G06F3/017 ,  G06F3/0334 ,  G06V10/255 ,  G06V40/20 ,  G16Y20/40 ,  H04N13/111

摘要： A virtual interface application presented in augmented reality (AR) is described for controlling Internet of Things (IoT) products. The virtual interface application enables a user of a portable electronic device to activate a virtual control interface overlay on a display, receive a selection from the user using her hands or feet, and send a control signal to a nearby IoT product which executes an action in accordance with the selection. The virtual control interface is presented on the display as a virtual object in an AR environment. The virtual interface application includes a foot tracking tool for detecting an intersection between the foot location (in the physical environment) and the virtual surface position (in the AR environment). When an intersection is detected, the virtual interface application sends a control signal with instructions to the IoT product.

公开(公告)号：US12045383B2

公开(公告)日：2024-07-23

申请号：US17901568

申请日：2022-09-01

申请人： Lei Zhang ,  Youjean Cho ,  Daekun Kim ,  Ava Robinson ,  Yu Jiang Tham ,  Rajan Vaish ,  Andrés Monroy-Hernández

发明人： Lei Zhang ,  Youjean Cho ,  Daekun Kim ,  Ava Robinson ,  Yu Jiang Tham ,  Rajan Vaish ,  Andrés Monroy-Hernández

IPC分类号： G06F3/01 ,  G06F1/16 ,  G06T19/00

CPC分类号： G06F3/011 ,  G06F1/163 ,  G06T19/006

摘要： Described are virtual AR interfaces for generating a virtual rotational interface for the purpose of controlling connected IoT devices using the inertial measurement unit (IMU) of a portable electronic device. The IMU control application enables a user of a portable electronic device to activate a virtual rotational interface overlay on a display and adjust a feature of a connected IoT product by rotating a portable electronic device. The device IMU moves a slider on the virtual rotational interface. The IMU control application sends a control signal to the IoT product which executes an action in accordance with the slider position. The virtual rotational interface is presented on the display as a virtual object in an AR environment. The IMU control application detects the device orientation (in the physical environment) and in response presents a corresponding slider element on the virtual rotational interface (in the AR environment).

公开(公告)号：US20240078759A1

公开(公告)日：2024-03-07

申请号：US17901518

申请日：2022-09-01

申请人： Daekun Kim ,  Lei Zhang ,  Youjean Cho ,  Ava Robinson ,  Yu Jiang Tham ,  Rajan Vaish ,  Andrés Monroy-Hernández

发明人： Daekun Kim ,  Lei Zhang ,  Youjean Cho ,  Ava Robinson ,  Yu Jiang Tham ,  Rajan Vaish ,  Andrés Monroy-Hernández

IPC分类号： G06T19/00 ,  G06T13/40 ,  G06T17/20

CPC分类号： G06T19/006 ,  G06T13/40 ,  G06T17/20 ,  G06T19/003

摘要： Multi-player co-located AR experiences are augmented by assigning characters and costumes to respective participants (a.k.a. “users” of AR-enabled mobile devices) in multi-player AR sessions for storytelling, play acting, and the like. Body tracking technology and augmented reality (AR) software are used to decorate the bodies of the co-located participants with virtual costumes within the context of the multi-player co-located AR experiences. Tracked bodies are distinguished to determine which body belongs to which user and hence which virtual costume belongs to which tracked body so that corresponding costumes may be assigned for display in augmented reality. A host-guest mechanism is used for networked assignment of characters and corresponding costumes in the co-located multi-player AR session. Body tracking technology is used to move the costume with the body as movement of the assigned body is detected.

公开(公告)号：US20240077934A1

公开(公告)日：2024-03-07

申请号：US17901568

申请日：2022-09-01

申请人： Lei Zhang ,  Youjean Cho ,  Daekun Kim ,  Ava Robinson ,  Yu Jiang Tham ,  Rajan Vaish ,  Andrés Monroy-Hernández

发明人： Lei Zhang ,  Youjean Cho ,  Daekun Kim ,  Ava Robinson ,  Yu Jiang Tham ,  Rajan Vaish ,  Andrés Monroy-Hernández

IPC分类号： G06F3/01 ,  G06F1/16 ,  G06T19/00

CPC分类号： G06F3/011 ,  G06F1/163 ,  G06T19/006

摘要： Described are virtual AR interfaces for generating a virtual rotational interface for the purpose of controlling connected IoT devices using the inertial measurement unit (IMU) of a portable electronic device. The IMU control application enables a user of a portable electronic device to activate a virtual rotational interface overlay on a display and adjust a feature of a connected IoT product by rotating a portable electronic device. The device IMU moves a slider on the virtual rotational interface. The IMU control application sends a control signal to the IoT product which executes an action in accordance with the slider position. The virtual rotational interface is presented on the display as a virtual object in an AR environment. The IMU control application detects the device orientation (in the physical environment) and in response presents a corresponding slider element on the virtual rotational interface (in the AR environment).

公开(公告)号：US11383156B1

公开(公告)日：2022-07-12

申请号：US17136843

申请日：2020-12-29

申请人： Andrés Monroy-Hernández ,  Ava Robinson ,  Yu Jiang Tham ,  Rajan Vaish

发明人： Andrés Monroy-Hernández ,  Ava Robinson ,  Yu Jiang Tham ,  Rajan Vaish

IPC分类号： A63F13/213 ,  A63F13/2145

摘要： Systems, methods, and computer readable media directed to multi-user visual experiences such as interactive gaming experiences and artistic media experiences. A viewing electronic device includes a camera configured to capture images, a display, and a processor coupled to the camera and the display. The processor is configured to capture, with the camera, images of a rotating marker where the rotating marker is presented on a monitor of a remote device and to present, on the display, a visual experience where the visual experience has an adjustable feature. The processor detects a parameter of the rotating marker from the captured images that corresponds to the adjustable feature and updates the visual experience responsive to the detected parameter. The parameter may be one or more of an angle, speed of rotation, direction of rotation, color, or pattern of the rotating marker.

公开(公告)号：US20240056524A1

公开(公告)日：2024-02-15

申请号：US18384251

申请日：2023-10-26

申请人： Andrés Monroy-Hernández ,  Sven Kratz ,  Rajan Vaish

发明人： Andrés Monroy-Hernández ,  Sven Kratz ,  Rajan Vaish

IPC分类号： H04M1/72469 ,  H04M1/72439 ,  A61L9/012 ,  G06F3/01 ,  G06F3/041 ,  G06F3/0488 ,  G06T11/00

CPC分类号： H04M1/72469 ,  H04M1/72439 ,  A61L9/012 ,  G06F3/011 ,  G06F3/0416 ,  G06F3/0488 ,  G06T11/00 ,  A61L2209/11 ,  G06T2200/24

摘要： An olfactory sticker used in chats between electronic devices to make messaging more immersive, personalized, and authentic by integrating olfactory information with traditional text-based formats and AR messaging. Olfactory stickers are used to indicate to a recipient that a message includes olfactory information. The olfactory sticker illustrates a graphical representation of a particular scent that can be sent and received via a chat or an AR message. Olfactory stickers provide the recipient control of accessing the transmitted scent at a desired time. This is particularly useful since certain olfactory information, i.e., scents, can be very direct and intrusive. Olfactory stickers are activated (i.e., release their scent) when they are tapped or rubbed by the recipient.

公开(公告)号：US20240027394A1

公开(公告)日：2024-01-25

申请号：US17868445

申请日：2022-07-19

申请人： Sven Kratz ,  Andrés Monroy-Hernández ,  Rajan Vaish

发明人： Sven Kratz ,  Andrés Monroy-Hernández ,  Rajan Vaish

IPC分类号： G01N27/414 ,  H04M1/72403

CPC分类号： G01N27/414 ,  H04M1/72403

摘要： An electronic device including an olfactory detector including an array of olfactory sensors for determining scents proximate the electronic device, such as a smartphone, smart eyewear, and a smart watch. Each sensor in the array of olfactory sensors is tuned to detect the presence and concentration of one or more specific chemical compounds or molecules. A fan creates airflow of ambient air across the olfactory sensors. An analog to digital (A/D) converter receives and processes the sensor outputs of the olfactory sensors and provides the processed sensor output to a processor for further processing. Scent type and intensity can be classified by using the information from the scent sensors as input for a machine learning model, generated through supervised training using labeled example measurements from our sensor array. The processor may display information of the determined scents on a display of the smart device, and the processor can also send information indicative of the determined scents to another device.

公开(公告)号：US11811964B1

公开(公告)日：2023-11-07

申请号：US17868395

申请日：2022-07-19

申请人： Andrés Monroy-Hernández ,  Sven Kratz ,  Rajan Vaish

发明人： Andrés Monroy-Hernández ,  Sven Kratz ,  Rajan Vaish

IPC分类号： H04M1/72469 ,  G06F3/041 ,  G06F3/01 ,  G06F3/0488 ,  G06T11/00 ,  A61L9/012 ,  H04M1/72439

CPC分类号： H04M1/72469 ,  A61L9/012 ,  G06F3/011 ,  G06F3/0416 ,  G06F3/0488 ,  G06T11/00 ,  H04M1/72439 ,  A61L2209/11 ,  G06T2200/24

摘要： An olfactory sticker used in chats between electronic devices to make messaging more immersive, personalized, and authentic by integrating olfactory information with traditional text-based formats and AR messaging. Olfactory stickers are used to indicate to a recipient that a message includes olfactory information. The olfactory sticker illustrates a graphical representation of a particular scent that can be sent and received via a chat or an AR message. Olfactory stickers provide the recipient control of accessing the transmitted scent at a desired time. This is particularly useful since certain olfactory information, i.e., scents, can be very direct and intrusive. Olfactory stickers are activated (i.e., release their scent) when they are tapped or rubbed by the recipient.

公开(公告)号：US20240087257A1

公开(公告)日：2024-03-14

申请号：US18507293

申请日：2023-11-13

申请人： Fannie Liu ,  Rajan Vaish

发明人： Fannie Liu ,  Rajan Vaish

IPC分类号： G06T19/00 ,  G06F3/04842 ,  G06F3/04845 ,  G08B5/22 ,  H04N5/91

CPC分类号： G06T19/006 ,  G06F3/04842 ,  G06F3/04845 ,  G08B5/22 ,  H04N5/91 ,  G06T2200/24 ,  G06T2219/024

摘要： Systems and methods enable users to engage in meaningful, authentic, online interactions by extracting objects (virtual or real) from an image or video and transferring the extracted objects into a real-world environment of another user in three-dimensional augmented reality. An object to be sent from the first user to the second user is generated using a drawing application, extracted from an image or a video, or an AR object is captured. A video is recorded showing the object responding to an action of the first user and metadata is generated relating to the response of the object to the action of the first user. The video and metadata are sent to the second user. Upon receipt, the video is watched by the second user and the object is selected for presentation in a display environment of the second user. The state of the object is preserved by the metadata.

公开(公告)号：US20240070299A1

公开(公告)日：2024-02-29

申请号：US17900792

申请日：2022-08-31

申请人： Youjean Cho ,  Chen Ji ,  Fannie Liu ,  Andrés Monroy-Hernández ,  Tsung-Yu Tsai ,  Rajan Vaish

发明人： Youjean Cho ,  Chen Ji ,  Fannie Liu ,  Andrés Monroy-Hernández ,  Tsung-Yu Tsai ,  Rajan Vaish

IPC分类号： G06F21/62 ,  G06T19/20

CPC分类号： G06F21/6209 ,  G06T19/20 ,  G06F2221/2113 ,  G06F2221/2137 ,  G06T2219/024

摘要： A collaborative session (e.g., a virtual time capsule) in which access to a collaborative object and added virtual content is selectively provided to participants/users. In one example of the collaborative session, a processor provides users with access to a collaborative object using respective physically remote devices, and associates virtual content received from the users with the collaborative object during a collaboration period. The processor maintains a timer including a countdown indicative of when the collaboration period ends for associating virtual content with the collaborative object. The processor provides the users with access to the collaborative object with associated virtual content at the end of the collaboration period.