公开(公告)号：US12192740B2

公开(公告)日：2025-01-07

申请号：US18532679

申请日：2023-12-07

Applicant: Snap Inc.

Inventor： Ilteris Canberk ,  Shin Hwun Kang

IPC: H04S7/00 ,  G02B27/01

Abstract: Devices, media, and methods are presented for an immersive augmented reality (AR) experience using an eyewear device with spatial audio. The eyewear device has a processor, a memory, an image sensor, and a speaker system. The eyewear device captures image information for an environment surrounding the device and identifies an object location within the same environment. The eyewear device then associates a virtual object with the identified object location. The eyewear device monitors the position of the device with respect to the virtual object and presents audio signals to alert the user that the identified object is in the environment.

公开(公告)号：US12086324B2

公开(公告)日：2024-09-10

申请号：US17550679

申请日：2021-12-14

Applicant: Snap Inc.

Inventor： Ilteris Canberk ,  Viktoria Hwang ,  Shin Hwun Kang ,  David Meisenholder ,  Daniel Moreno

IPC: G06F3/04815 ,  G02B27/01 ,  G06F3/01 ,  G06F3/04847 ,  G06T19/00 ,  G06V20/40 ,  G06V40/10 ,  G06V40/20

CPC classification number: G06F3/017 ,  G02B27/0101 ,  G02B27/017 ,  G06F3/04815 ,  G06F3/04847 ,  G06T19/006 ,  G06V20/46 ,  G06V40/113 ,  G06V40/28 ,  G02B2027/0138 ,  G02B2027/0178

Abstract: Example systems, devices, media, and methods are described for controlling virtual elements or graphical elements on a display in response to hand gestures detected by an eyewear device that is capturing frames of video data with its camera system. An image processing system detects a series of hand shapes in the video data and determines whether it matches a predefined series of hand gestures. Each predefined series of hand gestures is associated with an action. The system controls movement of the virtual element, relative to the display, in accordance with the associated action. In an example hand shape that includes a thumb sliding along an extended finger, the system establishes a finger scale along the extended finger, calibrates a graphical scale with the finger scale, and controls movement of an interactive graphical element, such as a slider, according to the current thumb position relative to the calibrated graphical scale.

公开(公告)号：US12008153B2

公开(公告)日：2024-06-11

申请号：US17988526

申请日：2022-11-16

Applicant: Snap Inc.

Inventor： Shin Hwun Kang ,  Ilteris Canberk ,  James Powderly ,  Dmytro Kucher ,  Dmytro Hovorov

IPC: A63F13/46 ,  A63F13/577 ,  G02B27/01 ,  G06F3/01 ,  G06F3/16 ,  G06T7/73

CPC classification number: G06F3/011 ,  A63F13/46 ,  A63F13/577 ,  G02B27/0101 ,  G02B27/0172 ,  G02B27/0179 ,  G06F3/167 ,  G06T7/73 ,  A63F2300/8082 ,  G02B2027/0138 ,  G02B2027/0178 ,  G02B2027/0187 ,  G06T2207/30208

Abstract: Interactive augmented reality experiences with an eyewear device including a position detection system and a display system. The eyewear device registers a first marker position for a user-controlled virtual game piece and a second marker for an interaction virtual game piece. The eyewear device monitors its position (e.g., location and orientation) and updates the position of the user-controlled virtual game piece accordingly. The eyewear device additionally monitors the position of the user-controlled virtual game piece with respect to the interaction virtual game piece for use in generating a score. Augmented reality examples include a “spheroidal balancing” augmented reality experience and a “spheroidal balancing” augmented reality experience.

公开(公告)号：US11863963B2

公开(公告)日：2024-01-02

申请号：US17528713

申请日：2021-11-17

Applicant: Snap Inc.

Inventor： Ilteris Canberk ,  Shin Hwun Kang

IPC: H04S7/00 ,  G02B27/01

CPC classification number: H04S7/303 ,  G02B27/017 ,  G02B2027/0138 ,  G02B2027/0178 ,  H04S2400/11

Abstract: Devices, media, and methods are presented for an immersive augmented reality (AR) experience using an eyewear device with spatial audio. The eyewear device has a processor, a memory, and image sensor, and a speaker system. The eyewear device captures image information for an environment surrounding the device and identifies an object location within the same environment. The eyewear device then associates a virtual object with the identified object location. The eyewear device monitors the position of the device with respect to the virtual object and presents audio signals to alert the user that the identified object is in the environment.

公开(公告)号：US11854147B2

公开(公告)日：2023-12-26

申请号：US17682296

申请日：2022-02-28

Applicant: Snap Inc.

Inventor： Shin Hwun Kang ,  Dmytro Kucher ,  Dmytro Hovorov ,  Ilteris Canberk

IPC: G06T19/00 ,  G06T7/73 ,  G02B27/01

CPC classification number: G06T19/003 ,  G02B27/0172 ,  G02B27/0176 ,  G06T7/73 ,  G02B2027/014 ,  G02B2027/0138 ,  G06T2207/30204

Abstract: Augmented reality guidance for guiding a user through an environment using an eyewear device. The eyewear device includes a display system and a position detection system. A user is guided though an environment by monitoring a current position of the eyewear device within the environment, identifying marker positions within a threshold of the current position, the marker positions defined with respect to the environment and associated with guidance markers, registering the marker positions, generating overlay image including the guidance markers, and presenting the overlay image on a display of the eyewear device.

公开(公告)号：US20230412650A1

公开(公告)日：2023-12-21

申请号：US18238364

申请日：2023-08-25

Applicant: Snap Inc.

Inventor： Kristian Bauer ,  Tiago Rafael Duarte ,  Terek Judi ,  Shin Hwun Kang ,  Karen Stolzenberg

IPC: H04L65/1069 ,  G02B27/01 ,  G06F3/14

CPC classification number: H04L65/1069 ,  G02B27/0172 ,  G06F3/1454 ,  G02B2027/0138 ,  G02B2027/0178

Abstract: A head-worn device system includes one or more cameras, one or more display devices and one or more processors. The system also includes a memory storing instructions that, when executed by the one or more processors, configure the system to perform operations to initiate or join a joint visual computing session. The method may comprise receiving user input to initiate a joint session of a visual computing experience, monitoring for short-range data transmissions including data indicating the existence of a current session of the visual computing experience, and based on determining that a current session is in process, providing a user input option to join the current session of the visual computing experience.

公开(公告)号：US20230154445A1

公开(公告)日：2023-05-18

申请号：US17454949

申请日：2021-11-15

Applicant: Snap Inc.

Inventor： Kristian Bauer ,  Tiago Rafael Duarte ,  Terek Judi ,  Shin Hwun Kang ,  Karen Stolzenberg

IPC: G10H1/00 ,  G10H1/46 ,  G02B27/01 ,  G06T19/00

CPC classification number: G10H1/0008 ,  G02B27/017 ,  G02B27/0179 ,  G06T19/006 ,  G10H1/46 ,  G02B2027/0187 ,  G10H2220/081 ,  G10H2220/106 ,  G10H2220/321 ,  G10H2240/175

Abstract: Disclosed is a method of providing a music creation interface using a head-mounted device, including displaying first and second geometric loops fixed relative to a location in the real world, the first and second geometric loops each including a plurality of beat indicators. The second geometric loop is spaced apart from the first geometric loop. An interface comprising a plurality of sound or note icons is displayed, and in response to receiving user selection to move a selected sound or note icon to a particular beat indicator on one of the geometric loops, the selected sound or note icon is displayed at the particular beat indicator. In use, the geometric loops are rotated relative to at least one play indicator, and the selected sound or note icon is rendered when it reaches the at least one play indicator.

公开(公告)号：US20220206588A1

公开(公告)日：2022-06-30

申请号：US17550679

申请日：2021-12-14

Applicant: Snap Inc.

Inventor： Ilteris Canberk ,  Viktoria Hwang ,  Shin Hwun Kang ,  David Meisenholder ,  Daniel Moreno

IPC: G06F3/01 ,  G06F3/04815 ,  G06F3/04847 ,  G06V40/20 ,  G06V40/10 ,  G06V20/40 ,  G02B27/01 ,  G06T19/00

Abstract: Example systems, devices, media, and methods are described for controlling virtual elements or graphical elements on a display in response to hand gestures detected by an eyewear device that is capturing frames of video data with its camera system. An image processing system detects a series of hand shapes in the video data and determines whether it matches a predefined series of hand gestures. Each predefined series of hand gestures is associated with an action. The system controls movement of the virtual element, relative to the display, in accordance with the associated action. In an example hand shape that includes a thumb sliding along an extended finger, the system establishes a finger scale along the extended finger, calibrates a graphical scale with the finger scale, and controls movement of an interactive graphical element, such as a slider, according to the current thumb position relative to the calibrated graphical scale.

公开(公告)号：US20210407205A1

公开(公告)日：2021-12-30

申请号：US17362377

申请日：2021-06-29

Applicant: Snap Inc.

Inventor： Ilteris Canberk ,  Shin Hwun Kang ,  Dmytro Kucher

IPC: G06T19/00 ,  H04N13/332 ,  G06K9/00 ,  G06T7/73 ,  H04N13/207 ,  H04N13/111 ,  G10L15/26 ,  G02B27/01

Abstract: Eyewear presenting text corresponding to spoken words (e.g., in speech bubbles) and optionally translating from one language to another. In one example, an interactive augmented reality experience is provided between two users of eyewear devices to allow one user of an eyewear device to share a personal attribute of the user with a second user. The personal attribute can be speech spoken by a remote second user of eyewear converted to text. The converted text can be displayed on a display of eyewear of the first user proximate the viewed second user. The personal attribute may be displayed in a speech bubble proximate the second user, such as proximate the head or mouth of the second user. The language of the spoken speech can be recognized by the second user eyewear, and translated to a language that is understood by the first user. In another example, the spoken words of a remote person is captured by the eyewear of a user, the position of the remote person is identified, the spoken word are converted to text, and the text is displayed (e.g., in a speech bubble) on an AR display of the eyewear adjacent the remote person.

公开(公告)号：US12061842B2

公开(公告)日：2024-08-13

申请号：US17657840

申请日：2022-04-04

Applicant: Snap Inc.

Inventor： Ilteris Kaan Canberk ,  Shin Hwun Kang

IPC: G06F3/16 ,  G06T3/40 ,  G06T7/70 ,  G06T19/00 ,  G10L15/22

CPC classification number: G06F3/167 ,  G06T3/40 ,  G06T7/70 ,  G06T19/006 ,  G10L15/22 ,  G10L2015/223 ,  G10L2015/228

Abstract: Disclosed are systems and methods for voice-based control of augmented reality (AR) objects on a wearable device. The systems and methods perform operations comprising: instructing a display element of the AR wearable device to present a visual indicator representing a cursor; receiving voice input representing a first virtual object; determining a real-world position within a real-world environment being viewed through the AR wearable device based on a current position of the visual indicator; and instructing the display element of the AR wearable device to present the first virtual object within the real-world environment at the real-world position.