Display device sharing and interactivity

    公开(公告)号:US11733956B2

    公开(公告)日:2023-08-22

    申请号:US17266912

    申请日:2019-09-03

    Applicant: Apple Inc.

    CPC classification number: G06F3/1454 G06F3/011 G06F3/017 G06F3/04815

    Abstract: In one implementation, a method of providing display device sharing and interactivity in simulated reality is performed at a first electronic device including one or more processors and a non-transitory memory. The method includes obtaining a gesture input to a first display device in communication with the first electronic device from a first user, where the first display device includes a first display. The method further includes transmitting a representation of the first display to a second electronic device in response to obtaining the gesture input. The method additionally includes receiving an input message directed to the first display device from the second electronic device, where the input message includes an input directive obtained by the second electronic device from a second user. The method also includes transmitting the input message to the first display device for execution by the first display device.

    AUTOMATIC MEASUREMENTS BASED ON OBJECT CLASSIFICATION

    公开(公告)号:US20230094061A1

    公开(公告)日:2023-03-30

    申请号:US18072855

    申请日:2022-12-01

    Applicant: Apple Inc.

    Abstract: Various implementations disclosed herein include devices, systems, and methods that provide measurements of objects based on a location of a surface of the objects. An exemplary process may include obtaining a three-dimensional (3D) representation of a physical environment that was generated based on depth data and light intensity image data, generating a 3D bounding box corresponding to an object in the physical environment based on the 3D representation, determining a class of the object based on the 3D semantic data, determining a location of a surface of the object based on the class of the object, the location determined by identifying a plane within the 3D bounding box having semantics in the 3D semantic data satisfying surface criteria for the object, and providing a measurement of the object, the measurement of the object determined based on the location of the surface of the object.

    Spatial video capture and replay
    13.
    发明授权

    公开(公告)号:US12254579B2

    公开(公告)日:2025-03-18

    申请号:US17984282

    申请日:2022-11-10

    Applicant: APPLE INC.

    Abstract: Various implementations disclosed herein include devices, systems, and methods that create a 3D video that includes determining first adjustments (e.g., first transforms) to video frames (e.g., one or more RGB images and depth images per frame) to align content in a coordinate system to remove the effects of capturing camera motion. Various implementations disclosed herein include devices, systems, and methods that playback a 3D video and includes determining second adjustments (e.g., second transforms) to remove the effects of movement of a viewing electronic device relative to a viewing environment during playback of the 3D video. Some implementations distinguish static content and moving content of the video frames to playback only moving objects or facilitate concurrent playback of multiple spatially related 3D videos. The 3D video may include images, audio, or 3D video of a video-capture-device user.

    PORTAL CONTENT FOR COMMUNICATION SESSIONS

    公开(公告)号:US20240404177A1

    公开(公告)日:2024-12-05

    申请号:US18679649

    申请日:2024-05-31

    Applicant: Apple Inc.

    Abstract: Devices, systems, and methods that provide a view of a three-dimensional (3D) environment (e.g., a viewer's room) with a portal providing views of another user's (e.g., a sender's) background environment during a communication session. For example, a process at a first electronic device may include presenting a view of a first 3D environment. Data representing a second 3D environment based at least in part on sensor data captured in a physical environment of a second electronic device may be obtained. Portal content based on the data representing the second 3D environment and a viewpoint within the first 3D environment may be determined. A portal with the portal content may be displayed in the view of the first 3D environment, where the portal content depicts a portion of the second 3D environment viewed through the portal from the viewpoint.

    CONTEXT-BASED AVATAR QUALITY
    16.
    发明公开

    公开(公告)号:US20240005622A1

    公开(公告)日:2024-01-04

    申请号:US18212247

    申请日:2023-06-21

    Applicant: Apple Inc.

    CPC classification number: G06T19/20 G06T15/005 G06T13/40 G06F3/013

    Abstract: Various implementations disclosed herein include devices, systems, and methods that provide a communication session in which a first device receives and uses streamed avatar data to render views that include a time-varying avatar, e.g., video content of some or all of another user sent from the other user's device during the communication session. In order to efficiently use resources (e.g., power, bandwidth, etc.), some implementations adapt the avatar provision process (e.g., video framerate, image resolution, etc.) based on user context, e.g., whether the viewer is looking at the avatar, whether the avatar is within the viewer's foveal region, or whether the avatar is within the viewer's field of view.

    Discovery Of And Connection To Remote Devices

    公开(公告)号:US20220114882A1

    公开(公告)日:2022-04-14

    申请号:US17557206

    申请日:2021-12-21

    Applicant: Apple Inc.

    Abstract: A method includes obtaining a location and a device type for one or more remote devices, and identifying one or more visible devices in one or more images, the one or more visible devices having a location and a device type. The method also includes matching a first visible device from the one or more visible devices with a first remote device from the one or more remote devices based on a location and a device type of the first visible device matching a location and a device type of the first remote device, obtaining a user input, and controlling a function of the first remote device based on the user input.

    LOCALIZATION FOR MOBILE DEVICES
    18.
    发明申请

    公开(公告)号:US20220101058A1

    公开(公告)日:2022-03-31

    申请号:US17493359

    申请日:2021-10-04

    Applicant: Apple Inc.

    Abstract: Systems and methods for localization for mobile devices are described. Some implementations may include accessing motion data captured using one or more motion sensors; determining, based on the motion data, a coarse localization, wherein the coarse localization includes a first estimate of position; obtaining one or more feature point maps, wherein the feature point maps are associated with a position of the coarse localization; accessing images captured using one or more image sensors; determining, based on the images, a fine localization pose by localizing into a feature point map of the one or more feature point maps, wherein the fine localization pose includes a second estimate of position and an estimate of orientation; generating, based on the fine localization pose, a virtual object image including a view of a virtual object; and displaying the virtual object image.

    Discovery of and connection to remote devices

    公开(公告)号:US11210932B2

    公开(公告)日:2021-12-28

    申请号:US16878249

    申请日:2020-05-19

    Applicant: Apple Inc.

    Abstract: A method includes identifying remote devices, at a host device, based on received signals that indicate locations and device types for the remote devices. The method also includes identifying visible devices in images of a location and matching a first visible device to a first remote device. The first visible device is matched with the first remote device based on presence of the first visible device within a search area of the images, the search area of the images is determined based on the location for the first remote device, the first visible device is matched with the first remote device based on the device type for the first remote device, and the first visible device is matched with the first remote device based on a machine recognizable indicator that is output by the first visible device. The method also includes pairing the first remote device with the host device.

    2D floorplan pipeline and refinement

    公开(公告)号:US12254244B1

    公开(公告)日:2025-03-18

    申请号:US17234898

    申请日:2021-04-20

    Applicant: Apple Inc.

    Abstract: Various implementations disclosed herein include devices, systems, and methods that determines two-dimensional (2D) floorplans based on image data and refines the 2D floorplans using various user interface controls. For example, an example process may include obtaining a floorplan that is generated based on a sequence of images of a physical environment, identify a type of an object depicted in the floorplan, identify a context of the object depicted in the floorplan, and modify the floorplan based on the type and the context of the object.

Patent Agency Ranking