Generating a view of an object from existing images

    公开(公告)号:US12136169B1

    公开(公告)日:2024-11-05

    申请号:US17582223

    申请日:2022-01-24

    Applicant: Apple Inc.

    Abstract: In some implementations, a method includes obtaining a request to view an object from a target point-of-view (POV). In some implementations, the object is represented in a plurality of images captured from corresponding POVs that are different from the target POV. In some implementations, the method includes generating respective contribution scores for the corresponding POVs indicative of respective contributions of the corresponding POVs to a view frustum of the target POV. In some implementations, the method includes determining a sequence in which the plurality of images is ordered based on the respective contribution scores for the corresponding POVs. In some implementations, the method includes synthesizing a new view of the object corresponding to the target POV by performing a warping operation to the plurality of images in accordance with the sequence.

    Methods and Devices for Improved Inverse Iterative Warping

    公开(公告)号:US20240303766A1

    公开(公告)日:2024-09-12

    申请号:US18664159

    申请日:2024-05-14

    Applicant: Apple Inc.

    CPC classification number: G06T3/18 G06T19/20 G06T2219/2021

    Abstract: In some implementations, a method includes: obtaining a reference image and forward flow information; identifying a neighborhood of pixels corresponding to a pixel within a target image based on the forward flow information; in accordance with a determination that a characterization vector for the neighborhood of pixels satisfies a background condition, generating a warp result for the pixel based on a first warp type; in accordance with a determination that the characterization vector satisfies a foreground condition, generating the warp result for the pixel based on a second warp type; and in accordance with a determination that the characterization vector does not satisfy the foreground or background conditions, generating the warp result for the pixel based on a third warp type; and populating pixel information for the pixel within the target image based on pixel information for a reference pixel within the reference image that corresponds to the warp result.

    RENDERING VIRTUAL CONTENT WITH IMAGE SIGNAL PROCESSING TONEMAPPING

    公开(公告)号:US20240378822A1

    公开(公告)日:2024-11-14

    申请号:US18658039

    申请日:2024-05-08

    Applicant: Apple Inc.

    Abstract: Various implementations disclosed herein include devices, systems, and methods that adjust a tone map used to display virtual content in an extended reality (XR) environment based a tone map of pass-through video. For example, a process may obtain virtual content associated with a virtual content tone map relating pixel luminance values to display space luminance values. The process further obtains pass-through video depicting a physical environment. The pass-through video is associated with an image signal processing (ISP) tone map relating pixel luminance values of the pass-through video signal to display space luminance values. The process further determines an adjustment adjusting the virtual content tone map based on the ISP tone map. The process further displays a view of an XR environment. The view includes the pass-through video displayed using the ISP tone map and the virtual content displayed using the virtual content tone map with the adjustment.

    Perspective Correction With Depth Clamping
    4.
    发明公开

    公开(公告)号:US20240233205A1

    公开(公告)日:2024-07-11

    申请号:US18612967

    申请日:2024-03-21

    Applicant: Apple Inc.

    CPC classification number: G06T11/00 G06T3/00 G06T7/50

    Abstract: In one implementation, a method of performing perspective correction of an image is performed by a device including an image sensor, a display, one or more processors, and non-transitory memory. The method includes capturing, using the image sensor, an image of a physical environment. The method includes obtaining a plurality of depths respectively associated with a plurality of pixels of the image of the physical environment. The method includes generating a clamped depth map of the image of the physical environment based on the plurality of depths, wherein each element of the clamped depth map has a depth value above or equal to a depth threshold. The method includes generating a display image by transforming, using the one or more processors, the image of the physical environment based on the clamped depth map and a difference between a perspective of the image sensor and a perspective of a user. The method includes displaying, on the display, the display image.

    DISPLAYING APPLICATIONS IN 3D WITHIN AN EXTENDED REALITY ENVIRONMENT

    公开(公告)号:US20250111623A1

    公开(公告)日:2025-04-03

    申请号:US18883108

    申请日:2024-09-12

    Applicant: Apple Inc.

    Abstract: Various implementations disclosed herein include devices, systems, and methods that apply a 3-dimensional (3D) effect to content for rendering. For example, a process may obtain content to render within an extended reality (XR) environment. The process may further generate, via a rendering framework, a two-dimensional (2D) rendering of the content The rendering framework generates 3D information based on the content. The process may further generate a 3D effect for rendering the content based on the 3D information. The process may further determine a location of a display region for the content within the XR environment and a view of the XR environment may be presented. Rendering of the content may be presented with the 3D effect at the location in the view of the XR environment.

    METHODS, SYSTEMS, AND APPARATUSES FOR MAINTAINING STEREO CONSISTENCY

    公开(公告)号:US20250071255A1

    公开(公告)日:2025-02-27

    申请号:US18773732

    申请日:2024-07-16

    Applicant: Apple Inc.

    Abstract: Various examples disclosed herein maintain stereo consistency in extended reality (XR) environments when receiving content data with a content frame depicting both a left eye portion of a content item corresponding to a first left eye viewpoint of the content item within the XR environment, and a right eye portion of the content item corresponding to a first right eye viewpoint of the content item within the XR environment. Stereo consistency may be maintained by determining to use an adjusted version of the content frame to provide a view of the content item wherein a left eye view from a second left eye viewpoint is different than the first left eye viewpoint and a right eye view from a second right eye viewpoint is different than the first right eye viewpoint, and presenting the left eye view and the right eye view based on the content frame and the adjustment.

    Passthrough Pipeline
    7.
    发明申请

    公开(公告)号:US20240404185A1

    公开(公告)日:2024-12-05

    申请号:US18672958

    申请日:2024-05-23

    Applicant: Apple Inc.

    Abstract: In one implementation, a method of pipelined blending an image with virtual content is performed by a device including an image sensor, a display, one or more processors, and non-transitory memory. The method includes capturing, with the image sensor, a first portion of an image of a physical environment. The method includes warping the first portion of the image of the physical environment to generate a warped first portion. The method includes blending the warped first portion with a first portion of virtual content to generate a blended first portion. The method includes displaying, on the display, the blended first portion. The method includes capturing, with the image sensor, a second portion of the image of the physical environment. The method includes warping the second portion of the image of the physical environment to generate a warped second portion. The method includes blending the warped second portion with a second portion of the virtual content to generate a blended second portion. The method includes displaying, on the display, the blended second portion, wherein displaying the blended first portion is initiated before blending the warped second portion is completed.

    Warping a Frame based on Pose and Warping Data

    公开(公告)号:US20240412320A1

    公开(公告)日:2024-12-12

    申请号:US18694355

    申请日:2022-09-08

    Applicant: Apple Inc.

    Abstract: In some implementations, a device includes an environmental sensor, a display, a non-transitory memory and one or more processors coupled with the environmental sensor, the display and the non-transitory memory. In some implementations, a method includes generating, at a first time, intermediate warping data for a warping operation to be performed on an application frame. In some implementations, the method includes obtaining, at a second time that occurs after the first time, via the environmental sensor, environmental data that indicates a pose of the device within a physical environment of the device. In some implementations, the method includes generating a warped application frame by warping the application frame in accordance with the pose of the device and the intermediate warping data. In some implementations, the method includes displaying the warped application frame on the display.

Patent Agency Ranking