3D reconstruction using wide-angle imaging devices

    公开(公告)号:US11335022B2

    公开(公告)日:2022-05-17

    申请号:US16897749

    申请日:2020-06-10

    申请人: Snap Inc.

    IPC分类号: G06T7/70 G06T3/00 G06T5/00

    摘要: The pose of a wide-angle image is determined by dewarping regions of the wide-angle image, determining estimated poses of the dewarped regions of the wide-angle image and deriving a pose of the wide-angle image from the estimated poses of the of the dewarped regions. The estimated poses of the dewarped regions may be determined by comparing features in the dewarped regions with features in prior dewarped regions from one or more prior wide-angle images, as well as by comparing features in the dewarped regions with features in a point cloud.

    Localizing an augmented reality device

    公开(公告)号:US11276201B1

    公开(公告)日:2022-03-15

    申请号:US15929976

    申请日:2020-06-01

    申请人: Snap Inc.

    IPC分类号: G06T7/73 G06T19/00

    摘要: Determining the position and orientation (or “pose”) of an augmented reality device includes capturing an image of a scene having a number of features and extracting descriptors of features of the scene represented in the image. The descriptors are matched to landmarks in a 3D model of the scene to generate sets of matches between the descriptors and the landmarks. Estimated poses are determined from at least some of the sets of matches between the descriptors and the landmarks. Estimated poses having deviations from an observed location measurement that are greater than a threshold value may be eliminated. Features used in the determination of estimated poses may also be weighted by the inverse of the distance between the feature and the device, so that closer features are accorded more weight.

    TRACKING AN AUGMENTED REALITY DEVICE
    4.
    发明公开

    公开(公告)号:US20230368476A1

    公开(公告)日:2023-11-16

    申请号:US18221179

    申请日:2023-07-12

    申请人: Snap Inc.

    IPC分类号: G06T19/00 G06T7/70

    CPC分类号: G06T19/006 G06T7/70

    摘要: An augmented reality device generates an updated position and orientation (pose) value by initially determining, using image-based processing, a pose estimate from a current image frame, a previous image frame and a previous pose. An IMU pose is then obtained from data generated by an inertial measurement unit including motion sensors, and transformation between the pose estimate and the IMU pose is determined. The transformation is applied to the pose estimate to generate a pose update. Media content is applied to a position within an image, the position being based on the pose update.

    LOCALIZING AN AUGMENTED REALITY DEVICE

    公开(公告)号:US20220156976A1

    公开(公告)日:2022-05-19

    申请号:US17589523

    申请日:2022-01-31

    申请人: Snap Inc.

    IPC分类号: G06T7/73 G06T19/00

    摘要: Determining the position and orientation (or “pose”) of an augmented reality device includes capturing an image of a scene having a number of features and extracting descriptors of features of the scene represented in the image. The descriptors are matched to landmarks in a 3D model of the scene to generate sets of matches between the descriptors and the landmarks. Estimated poses are determined from at least some of the sets of matches between the descriptors and the landmarks. Estimated poses having deviations from an observed location measurement that are greater than a threshold value may be eliminated. Features used in the determination of estimated poses may also be weighted by the inverse of the distance between the feature and the device, so that closer features are accorded more weight.

    3D RECONSTRUCTION USING WIDE-ANGLE IMAGING DEVICES

    公开(公告)号:US20210390729A1

    公开(公告)日:2021-12-16

    申请号:US16897749

    申请日:2020-06-10

    申请人: Snap Inc.

    IPC分类号: G06T7/70 G06T5/00 G06T3/00

    摘要: The pose of a wide-angle image is determined by dewarping regions of the wide-angle image, determining estimated poses of the dewarped regions of the wide-angle image and deriving a pose of the wide-angle image from the estimated poses of the of the dewarped regions. The estimated poses of the dewarped regions may be determined by comparing features in the dewarped regions with features in prior dewarped regions from one or more prior wide-angle images, as well as by comparing features in the dewarped regions with features in a point cloud.

    Tracking an augmented reality device

    公开(公告)号:US11748954B2

    公开(公告)日:2023-09-05

    申请号:US17083722

    申请日:2020-10-29

    申请人: Snap Inc.

    IPC分类号: G06T19/00 G06T7/70

    CPC分类号: G06T19/006 G06T7/70

    摘要: An augmented reality device generates an updated position and orientation (pose) value by initially determining, using image-based processing, a pose estimate from a current image frame, a previous image frame and a previous pose. An IMU pose is then obtained from data generated by an inertial measurement unit including motion sensors, and transformation between the pose estimate and the IMU pose is determined. The transformation is applied to the pose estimate to generate a pose update. Media content is applied to a position within an image, the position being based on the pose update.

    3D RECONSTRUCTION USING WIDE-ANGLE IMAGING DEVICES

    公开(公告)号:US20220230351A1

    公开(公告)日:2022-07-21

    申请号:US17714720

    申请日:2022-04-06

    申请人: Snap Inc.

    IPC分类号: G06T7/70 G06T3/00 G06T5/00

    摘要: The pose of a wide-angle image is determined by dewarping regions of the wide-angle image, determining estimated poses of the dewarped regions of the wide-angle image and deriving a pose of the wide-angle image from the estimated poses of the of the dewarped regions. The estimated poses of the dewarped regions may be determined by comparing features in the dewarped regions with features in prior dewarped regions from one or more prior wide-angle images, as well as by comparing features in the dewarped regions with features in a point cloud.

    TRACKING AN AUGMENTED REALITY DEVICE

    公开(公告)号:US20210375054A1

    公开(公告)日:2021-12-02

    申请号:US17083722

    申请日:2020-10-29

    申请人: Snap Inc.

    IPC分类号: G06T19/00 G06T7/70

    摘要: An augmented reality device generates an updated position and orientation (pose) value by initially determining, using image-based processing, a pose estimate from a current image frame, a previous image frame and a previous pose. An IMU pose is then obtained from data generated by an inertial measurement unit including motion sensors, and transformation between the pose estimate and the IMU pose is determined. The transformation is applied to the pose estimate to generate a pose update. Media content is applied to a position within an image, the position being based on the pose update.

    Localizing an augmented reality device

    公开(公告)号:US11854231B2

    公开(公告)日:2023-12-26

    申请号:US17589523

    申请日:2022-01-31

    申请人: Snap Inc.

    IPC分类号: G06T7/73 G06T19/00

    摘要: Determining the position and orientation (or “pose”) of an augmented reality device includes capturing an image of a scene having a number of features and extracting descriptors of features of the scene represented in the image. The descriptors are matched to landmarks in a 3D model of the scene to generate sets of matches between the descriptors and the landmarks. Estimated poses are determined from at least some of the sets of matches between the descriptors and the landmarks. Estimated poses having deviations from an observed location measurement that are greater than a threshold value may be eliminated. Features used in the determination of estimated poses may also be weighted by the inverse of the distance between the feature and the device, so that closer features are accorded more weight.