Marker-based guided AR experience
    23.
    发明授权

    公开(公告)号:US12108011B2

    公开(公告)日:2024-10-01

    申请号:US17566979

    申请日:2021-12-31

    申请人: Snap Inc.

    摘要: Systems, devices, media, and methods are presented for producing an augmented reality (AR) experience for display on a smart eyewear device. The AR production system includes a marker registration utility for setting and storing markers, a localization utility for locating the eyewear device relative to a marker location and to the mapped environment, and a virtual object rendering utility to presenting one or more virtual objects having a desired size, shape, and orientation. A high-definition camera captures an input image of the environment. If the input image includes a marker, the system retrieves from memory a set of data including a first marker location expressed in terms relative to a marker coordinate system. The localization utility determines a local position of the eyewear device relative to the marker location. The virtual object rendering utility prepares one or more virtual objects for display based on the eyewear location, the head pose of the wearer, and the location of one or more physical object landmarks in the environment.

    Computer vision and mapping for audio

    公开(公告)号:US12080261B2

    公开(公告)日:2024-09-03

    申请号:US18142181

    申请日:2023-05-02

    申请人: Snap Inc.

    IPC分类号: G10H1/00 G06T7/90

    摘要: Systems, devices, media, and methods are presented for playing audio sounds, such as music, on a portable electronic device using a digital color image of a note matrix on a map. A computer vision engine, in an example implementation, includes a mapping module, a color detection module, and a music playback module. The camera captures a color image of the map, including a marker and a note matrix. Based on the color image, the computer vision engine detects a token color value associated with each field. Each token color value is associated with a sound sample from a specific musical instrument. A global state map is stored in memory, including the token color value and location of each field in the note matrix. The music playback module, for each column, in order, plays the notes associated with one or more the rows, using the corresponding sound sample, according to the global state map.

    Dirty lens image correction
    26.
    发明授权

    公开(公告)号:US12073536B2

    公开(公告)日:2024-08-27

    申请号:US17883602

    申请日:2022-08-09

    申请人: Snap Inc.

    摘要: Systems and method for correcting images including artifacts due to dirty camera lenses of electronic device are disclosed. Correction of images by the systems and methods includes obtaining a first raw pixel image of a scene captured with a first camera, obtaining a second raw image of the scene captured with a second camera separate from the first camera in a camera baseline direction, rectifying the first and second raw pixel images to create respective first and second rectified pixel images, determining disparity correspondence between corresponding image pixel pairs of the first and second rectified images in the camera baseline direction, mapping first and second rectified images into the same domain using the determined disparity, detect image artifact regions within each domain mapped image by comparing corresponding regions of the domain mapped images, determining correction factors for each detected image artifact region, and correcting the rectified first and second images by applying the determined correction factors.