System and method for event camera data processing

    公开(公告)号:US11611715B2

    公开(公告)日:2023-03-21

    申请号:US17509354

    申请日:2021-10-25

    Applicant: Apple Inc.

    Abstract: In some embodiments, a method is performed at a device with a processor, non-transitory memory, and an event camera including pixel sensors distributed across an area. The method includes converting an event stream from a pixel sensor over a first time period into event frames by dividing the first time period into sub-periods, and binning pixel events of the event stream, where each of the sub-periods is associated with a frame sub-period identifier. The method further includes addressing the pixel sensors by sub-dividing the area into tiles, where each of the tiles includes a grouping of the pixel sensors, and a tile address of a particular pixel sensor is a combination of a tile identifier and a position locator of the particular pixel sensor. The method further includes encoding the pixel events as a function of a tile address, a frame sub-period identifier, and a brightness indicator value.

    GENERATING STATIC IMAGES WITH AN EVENT CAMERA

    公开(公告)号:US20200252533A1

    公开(公告)日:2020-08-06

    申请号:US16649005

    申请日:2018-06-28

    Applicant: Apple Inc.

    Abstract: In accordance with some embodiments, a method is performed at an image processing device with a processor and non-transitory memory. The method includes triggering light emission, over a first emission duration, having a characterizing intensity as a function of time. The method further includes obtaining respective pixel events, from an event camera, corresponding to reflections of the light emission during the first emission duration, each respective pixel event corresponding to a breach of a respective comparator threshold indicative of a brightness level, each respective pixel event characterized by a respective electrical threshold value and a timestamp at which the respective electrical threshold value was breached. The method also includes generating a static image by determining a plurality of luminance estimation values from the respective pixel events, wherein the plurality of luminance estimation values respectively correspond to an amount of reflected light received by portions of the event camera.

    Media Compositor For Computer-Generated Reality

    公开(公告)号:US20200043237A1

    公开(公告)日:2020-02-06

    申请号:US16533053

    申请日:2019-08-06

    Applicant: Apple Inc.

    Abstract: One implementation forms a composited stream of computer-generated reality (CGR) content using multiple data streams related to a CGR experience to facilitate recording or streaming. A media compositor obtains a first data stream of rendered frames and a second data stream of additional data. The rendered frame content (e.g., 3D models) represents real and virtual content rendered during a CGR experience at a plurality of instants in time. The additional data of the second data stream relates to the CGR experience, for example, relating to audio, audio sources, metadata identifying detected attributes of the CGR experience, image data, data from other devices involved in the CGR experience, etc. The media compositor forms a composited stream that aligns the rendered frame content with the additional data for the plurality of instants in time, for example, by forming time-stamped, n-dimensional datasets (e.g., images) corresponding to individual instants in time.

    Generating static images with an event camera

    公开(公告)号:US11019273B2

    公开(公告)日:2021-05-25

    申请号:US16649005

    申请日:2018-06-28

    Applicant: Apple Inc.

    Abstract: In accordance with some embodiments, a method is performed at an image processing device with a processor and non-transitory memory. The method includes triggering light emission, over a first emission duration, having a characterizing intensity as a function of time. The method further includes obtaining respective pixel events, from an event camera, corresponding to reflections of the light emission during the first emission duration, each respective pixel event corresponding to a breach of a respective comparator threshold indicative of a brightness level, each respective pixel event characterized by a respective electrical threshold value and a timestamp at which the respective electrical threshold value was breached. The method also includes generating a static image by determining a plurality of luminance estimation values from the respective pixel events, wherein the plurality of luminance estimation values respectively correspond to an amount of reflected light received by portions of the event camera.

    Electronic Device With Coordinated Camera and Display Operation

    公开(公告)号:US20250148730A1

    公开(公告)日:2025-05-08

    申请号:US19016230

    申请日:2025-01-10

    Applicant: Apple Inc.

    Abstract: An electronic device may have a camera and a display. The display may be configured to display virtual reality content for a user in which no real-world content from the camera is displayed or mixed reality content in which a combination of real-world content from the camera and overlaid virtual reality content is displayed. Control circuitry in the device may adjust the display and camera while transitioning between virtual reality and mixed reality modes. The control circuitry may reconfigure the camera to exhibit a desired frame rate immediately upon transitioning from virtual reality mode to mixed reality mode. Transitions between modes may be accompanied by smooth transitions between frame rates to avoid visible artifacts on the display. The camera frame rate may be synchronized to the display frame rate for at least part of the transition between the virtual reality and mixed reality modes.

    Object Localization System
    8.
    发明申请

    公开(公告)号:US20250076672A1

    公开(公告)日:2025-03-06

    申请号:US18952845

    申请日:2024-11-19

    Applicant: Apple Inc.

    Abstract: Fiducial patterns that produce 2D Barker code-like diffraction patterns at a camera sensor are etched or otherwise provided on a cover glass in front of a camera. 2D Barker code kernels, when cross-correlated with the diffraction patterns captured in images by the camera, provide sharp cross-correlation peaks. Misalignment of the cover glass with respect to the camera can be derived by detecting shifts in the location of the detected peaks with respect to calibrated locations. Devices that include multiple cameras behind a cover glass with one or more fiducials on the cover glass in front of each camera are also described. The diffraction patterns caused by the fiducials at the various cameras may be analyzed to detect movement or distortion of the cover glass in multiple degrees of freedom.

    MODIFYING RENDERED IMAGE DATA BASED ON AMBIENT LIGHT FROM A PHYSICAL ENVIRONMENT

    公开(公告)号:US20230290321A1

    公开(公告)日:2023-09-14

    申请号:US18101779

    申请日:2023-01-26

    Applicant: Apple Inc.

    CPC classification number: G09G5/10 G02B27/017 G06T19/006

    Abstract: A method includes obtaining rendered image data that includes a representation of an object for display using a see-through display. The see-through display permits ambient light from a physical environment through the see-through display. The method includes sensing a plurality of light superposition characteristic values associated with the ambient light that quantifies the ambient light. The method includes determining a plurality of display correction values associated with the electronic device based on the plurality of light superposition characteristic values and predetermined display characteristics of the representation of the object. The method includes generating, from the rendered image data, display data for the see-through display in accordance with the plurality of display correction values in order to satisfy the predetermined display characteristics of the representation of the object within a performance threshold.

Patent Agency Ranking