公开(公告)号：US11475538B2

公开(公告)日：2022-10-18

申请号：US17314217

申请日：2021-05-07

Applicant: GoPro, Inc.

Inventor： Balineedu Chowdary Adsumilli ,  Adeel Abbas

IPC: H04N5/232 ,  G06T3/40

Abstract: Systems and methods for providing panoramic image and/or video content using multi-resolution stitching. Panoramic content may include stitched spherical (360-degree) images and/or VR video. In some implementations, multi-resolution stitching functionality may be embodied in a spherical image capture device that may include two lenses configured to capture pairs of hemispherical images. The capture device may obtain images (e.g., representing left and right hemispheres) that may be characterized by 180-degree (or greater) field of view. Source images may be combined using multi-resolution stitching methodology. Source images may be transformed to obtain multiple image components characterized by two or more image resolutions. The stitched image may be encoded using selective encoding methodology including: partitioning source images into a low resolution/frequency and a high resolution/frequency components; stitching low resolution/frequency components using coarse stitching operation, stitching high resolution/high frequency components using a refined stitch operation; combining stitched LF components and stitched HF components.

公开(公告)号：US11228749B2

公开(公告)日：2022-01-18

申请号：US16866447

申请日：2020-05-04

Applicant: GoPro, Inc.

Inventor： Adeel Abbas ,  Balineedu Chowdary Adsumilli ,  David Newman

IPC: H04N13/111 ,  H04N13/194 ,  H04N13/243 ,  H04N13/344 ,  H04N13/161 ,  H04N13/282 ,  H04N5/232 ,  H04N19/597 ,  H04N5/225

Abstract: Apparatus and methods for characterizing panoramic content, such as by a wide field of view and large image size. In one implementation, a panoramic image may be mapped to a cube or any other projection e.g icosahedron or octahedron. The disclosure exploits content continuity between facets, such as in the case of encoding/decoding cube-projected images. One facet may be encoded/decoded independently from other facets to obtain a seed facet. One or more transformed versions of the seed facet may be obtained; e.g., one corresponding to a 90° counterclockwise rotation, another to a 90° clockwise rotation, and one to an 180° rotation. Transformed versions may be used to form an augmented image. The remaining facets of the cube may be encoded using transformed versions within the augmented image. Continuity between transformed facets in the top row of the augmented image and facets in the middle row of the augmented image may be utilized for motion or intra prediction, and to obtain greater encoding performance compared to encoding facets independently.

公开(公告)号：US20210321047A1

公开(公告)日：2021-10-14

申请号：US17240868

申请日：2021-04-26

Applicant: GoPro, Inc.

Inventor： Adeel Abbas ,  Cesar Douady ,  Timothy MacMillan

IPC: H04N5/265 ,  H04N7/18 ,  H04N5/232 ,  H04N5/247

Abstract: Apparatus and methods for stitching images, or re-stitching previously stitched images. Specifically, the disclosed systems in one implementation save stitching information and/or original overlap source data during an original stitching process. During subsequent retrieval, rendering, and/or display of the stitched images, the originally stitched image can be flexibly augmented, and/or re-stitched to improve the original stitch quality. Practical applications of the disclosed solutions enable, among other things, a user to create and stitch a wide field of view (FOV) panorama from multiple source images on a device with limited processing capability (such as a mobile phone or other capture device). Moreover, post-processing stitching allows for the user to convert from one image projection to another without fidelity loss (or with an acceptable level of loss).

公开(公告)号：US11049219B2

公开(公告)日：2021-06-29

申请号：US16234396

申请日：2018-12-27

Applicant: GOPRO, INC.

Inventor： David Newman ,  Daryl Stimm ,  Adeel Abbas

IPC: G06T3/40 ,  H04N19/80 ,  G06T3/00

Abstract: Methods and apparatus for multi-encoder processing of high resolution content. In one embodiment, the method includes capturing high resolution imaging content; splitting up the captured high resolution imaging content into respective portions; feeding the split up portions to respective imaging encoders; packing encoded content from the respective imaging encoders into an A/V container; and storing and/or transmitting the A/V container. In another embodiment, the method includes retrieving and/or receiving an A/V container; splitting up the retrieved and/or received A/V container into respective portions; feeding the split up portions to respective imaging decoders; stitching the decoded imaging portions into a common imaging portion; and storing and/or displaying at least a portion of the common imaging portion.

公开(公告)号：US10757423B2

公开(公告)日：2020-08-25

申请号：US16428786

申请日：2019-05-31

Applicant: GoPro, Inc.

Inventor： Adeel Abbas

IPC: H04N19/176 ,  H04N19/597 ,  H04N19/105 ,  H04N19/147 ,  H04N19/85

Abstract: Apparatus and methods for encoding panoramic content, such as by a wide field of view and large image size. In one implementation, a panoramic image may be mapped to a cube, equirectangular or any other projection e.g., icosahedron or octahedron. Projection may be selected adaptively based on evaluation of the panoramic content. Content evaluation may include obtaining rate distortion cost metric for a given projection configuration including projection type, projection arrangement, and projection orientation. Projection configuration with the lowest cost may be selected as target projection for encoding content. As content composition changes (e.g., object motion, texture presence and/or location) projection may be adaptively selected to match changes in the content. Adaptive content selection methodology may provide for a lower encoded bitrate for a given encoded quality and/or higher quality for a given bitrate.

公开(公告)号：US10728551B2

公开(公告)日：2020-07-28

申请号：US16143224

申请日：2018-09-26

Applicant: GoPro, Inc.

Inventor： Adeel Abbas ,  David Newman

IPC: H04N19/119 ,  H04N19/124 ,  H04N19/14 ,  H04N19/172 ,  H04N19/176

Abstract: Apparatus and methods for providing block-based layouts for non-rectangular regions between non-contiguous imaging regions in projections such as, for example, Rotated Sphere Projections (RSP). In one embodiment, methods and apparatus for selecting coding unit block sizes for non-rectangular regions disposed between non-contiguous imaging portions is disclosed. The methods and apparatus may be configured to obtain a frame of captured imaging content, the frame of captured imaging content including the non-rectangular regions disposed between the non-contiguous imaging portions; select a plurality of differing coding block sizes for each of a plurality of regions located within the non-rectangular regions disposed between the non-contiguous imaging portions; determine which one of the plurality of differing coding block sizes optimizes an encoding efficiency for the obtained frame of captured imaging content; and encode the obtained frame of captured imaging content in accordance with the determined one off the plurality of differing coding block sizes.

公开(公告)号：US20200221067A1

公开(公告)日：2020-07-09

申请号：US16823118

申请日：2020-03-18

Applicant: GoPro, Inc.

Inventor： Adeel Abbas ,  Timothy MacMillan

IPC: H04N13/243 ,  H04N5/232 ,  G06T3/00 ,  G06T7/38 ,  G06T7/73 ,  G06T7/593 ,  H04N19/136 ,  H04N19/44 ,  G06K9/46 ,  G06K9/52 ,  G06K9/62 ,  G06T3/20 ,  G06T7/00 ,  G06T11/60 ,  G06T15/20 ,  H04N13/302 ,  H04N13/344 ,  H04N13/10 ,  H04N13/334 ,  H04N13/282 ,  G06T7/55 ,  H04N5/265

Abstract: Apparatus and methods for providing a frame packing arrangement for the encoding/decoding of, for example, panoramic content. In one embodiment, the frame packing arrangement utilizes overlapping imaging data so as to enable, for example, a post-decode stitching operation to be performed. The frame packing arrangement may utilize a number of projection formats, such as a cubemap projection, and may utilize any number of differing aspect ratios such as, without limitation, 4×3, 3×2, 4×2, 2×4 aspect ratios. Additionally, the overlapping imaging data may be positioned within the frame packing arrangement chosen so as to improve upon coding efficiency. For example, the overlapping imaging data may be positioned within the frame packing arrangement so as to emphasize image continuity. An encoder apparatus and decoder apparatus for use with the aforementioned frame packing arrangements are also disclosed.

公开(公告)号：US20200177863A1

公开(公告)日：2020-06-04

申请号：US16786283

申请日：2020-02-10

Applicant: GoPro, Inc.

Inventor： Adeel Abbas ,  David A. Newman

IPC: H04N13/161 ,  H04N5/232 ,  G06T3/40 ,  H04N19/597 ,  H04N19/124 ,  H04N19/40 ,  H04N19/70 ,  H04N13/156 ,  H04N13/178 ,  H04N7/015

Abstract: Apparatus and methods for providing a frame packing arrangement for the encoding/decoding of, for example, panoramic content. In one embodiment, an encoder apparatus is disclosed. In a variant, the encoder apparatus is configured to encode Segmented Sphere Projections (SSP) imaging data and/or Rotated Sphere Projections (RSP) imaging data into an extant imaging format. In another variant, a decoder apparatus is disclosed. In one embodiment, the decoder apparatus is configured to decode SSP imaging data and/or RSP imaging data from an extant imaging format. Computing devices, computer-readable storage apparatus, integrated circuits and methods for using the aforementioned encoder and decoder are also disclosed.

公开(公告)号：US10652558B2

公开(公告)日：2020-05-12

申请号：US16234214

申请日：2018-12-27

Applicant: GoPro, Inc.

Inventor： Adeel Abbas ,  Balineedu Chowdary Adsumilli ,  David Newman

IPC: H04N19/30 ,  H04N19/187 ,  H04N19/167 ,  H04N19/29 ,  H04N19/146 ,  H04N19/59

Abstract: Apparatus and methods for digital video data compression via a scalable, multi-resolution approach. In one embodiment, the video content may be encoded using a multi-resolution and/or multi-quality scalable coding approach that reduces computational and/or energy load on a client device. In one implementation, a low fidelity image is obtained based on a first full resolution image. The low fidelity image may be encoded to obtain a low fidelity bitstream. A second full resolution image may be obtained based on the low fidelity bitstream. A portion of a difference image obtained based on the second full resolution image and the first full resolution may be encoded to obtain a high fidelity bitstream. The low fidelity bitstream and the high fidelity bitstream may be provided to e.g., a receiving device.

公开(公告)号：US10602191B2

公开(公告)日：2020-03-24

申请号：US15415603

申请日：2017-01-25

Applicant: GoPro, Inc.

Inventor： Adeel Abbas ,  David Newman

IPC: H04N19/86 ,  H04N19/597 ,  H04N19/85 ,  H04N19/82 ,  H04N5/341 ,  H04N5/232

Abstract: Panoramic content may be characterized by a wide field of view and large image size. Panoramic image may be mapped to cube projection. When encoding/decoding cube-projected images, the disclosure exploits content continuity between cube facets. One facet may be encoded/decoded independently from other facets to obtain a seed facet. One or more transformed versions of the seed facet may be obtained, for example, one corresponding to a 90° counterclockwise rotation, another to a 90° clockwise rotation, and one to an 180° rotation. Transformed versions may be used to form an augmented image. The remained facets of the cube may be encoded using transformed versions within the augmented image. Boundary filtering may be applied to one or more edges of one or more facets.