公开(公告)号：US20170142435A1

公开(公告)日：2017-05-18

申请号：US15354933

申请日：2016-11-17

Applicant: GoPro, Inc.

Inventor： David A. Newman ,  Brian G. Schunck

IPC: H04N19/44 ,  G11B20/00 ,  H04N19/63 ,  G11B27/031 ,  H04N19/423

CPC classification number: H04N19/44 ,  G11B20/00007 ,  G11B27/031 ,  G11B2020/00072 ,  H04N19/423 ,  H04N19/63

Abstract: A system and method disposed to enable encoding, decoding and manipulation of digital video with substantially less processing load than would otherwise required. In particular, one disclosed method is directed to generating a compressed video data structure that is selectively decodable to a plurality of resolutions including the full resolution of the uncompressed stream. The desired number of data components and the content of the data components that make up the compressed video data, which determine the available video resolutions, are variable based upon the processing carried out and the resources available to decode and process the data components. During decoding, efficiency is substantially improved because only the data components necessary to generate a desired resolution are decoded. In variations, both temporal and spatial decoding are utilized to reduce frame rates, and hence, further reduce processor load. The system and method are particularly useful for real-time video editing applications.

公开(公告)号：US09646652B2

公开(公告)日：2017-05-09

申请号：US14705864

申请日：2015-05-06

Applicant: GoPro, Inc.

Inventor： Balineedu Chowdary Adsumilli ,  Ryan Lustig ,  David A. Newman

IPC: H04N9/80 ,  H04N5/93 ,  G11B27/10 ,  G06K9/00 ,  G11B27/30 ,  G11B27/031 ,  G10L15/06 ,  G10L15/22 ,  H04N5/77 ,  H04N5/91 ,  H04N9/82 ,  G11B27/13 ,  G11B27/34 ,  G10L25/57 ,  G10L25/54

CPC classification number: G11B27/10 ,  G06K9/00718 ,  G06K9/00751 ,  G06K2009/00738 ,  G10L15/063 ,  G10L15/22 ,  G10L25/54 ,  G10L25/57 ,  G10L2015/0631 ,  G10L2015/223 ,  G11B27/031 ,  G11B27/13 ,  G11B27/28 ,  G11B27/3081 ,  G11B27/34 ,  H04N5/77 ,  H04N5/772 ,  H04N5/91 ,  H04N9/8205

Abstract: Video and corresponding metadata is accessed. Events of interest within the video are identified based on the corresponding metadata, and best scenes are identified based on the identified events of interest. In one example, best scenes are identified based on the motion values associated with frames or portions of a frame of a video. Motion values are determined for each frame and portions of the video including frames with the most motion are identified as best scenes. Best scenes may also be identified based on the motion profile of a video. The motion profile of a video is a measure of global or local motion within frames throughout the video. For example, best scenes are identified from portion of the video including steady global motion. A video summary can be generated including one or more of the identified best scenes.

公开(公告)号：US20160301846A1

公开(公告)日：2016-10-13

申请号：US15180529

申请日：2016-06-13

Applicant: GoPro, Inc.

Inventor： Paul Mobbs ,  Scott Patrick Campbell ,  David A. Newman ,  Kasturi Rangam ,  Sumit Chawla

IPC: H04N5/232 ,  H04N5/353

CPC classification number: H04N5/2329 ,  H04N1/2137 ,  H04N1/215 ,  H04N5/23229 ,  H04N5/23232 ,  H04N5/23277 ,  H04N5/3532 ,  H04N5/35572 ,  H04N2101/00

Abstract: An image sensor of a camera system captures an image over an image capture interval of time, and waits a blanking interval of time before capturing an additional image. The captured image is provided to a frame controller, and is buffered until an image signal processor accesses the captured image. The image signal processor processes the accessed image over an image processing interval of time, producing a processed image. The image processing interval of time is selected to be greater than the image capture interval of time, but less than the sum of the image capture interval of time and the blanking interval of time. By reducing the image capture interval of time but maintaining an image processing interval of time, rolling shutter artifacts are beneficially reduced without increasing the processing resources or power required by the image signal processor to process the image.

公开(公告)号：US20160205381A1

公开(公告)日：2016-07-14

申请号：US15077911

申请日：2016-03-23

Applicant: GoPro, Inc.

Inventor： Timothy Macmillan ,  Scott Patrick Campbell ,  David A. Newman ,  Yajie Sun

IPC: H04N13/02

CPC classification number: H04N13/246 ,  H04N5/23203 ,  H04N5/23238 ,  H04N5/247 ,  H04N5/265 ,  H04N5/3532 ,  H04N13/00 ,  H04N13/239 ,  H04N13/296 ,  H04N17/002

Abstract: A pair of cameras having an overlapping field of view is aligned based on images captured by image sensors of the pair of cameras. A pixel shift is identified between the images. Based on the identified pixel shift, a calibration is applied to one or both of the pair of cameras. To determine the pixel shift, the camera applies correlation methods including edge matching. Calibrating the pair of cameras may include adjusting a read window on an image sensor. The pixel shift can also be used to determine a time lag, which can be used to synchronize subsequent image captures.

Abstract translation: 具有重叠视野的一对相机基于由该对相机的图像传感器捕获的图像来对准。 在图像之间识别像素漂移。 基于所识别的像素偏移，校准被应用于该对照相机中的一个或两个。 为了确定像素偏移，相机应用包括边缘匹配的相关方法。 校准一对相机可能包括调整图像传感器上的读取窗口。 像素偏移也可用于确定时间滞后，可用于同步后续图像捕获。

公开(公告)号：US09338373B2

公开(公告)日：2016-05-10

申请号：US14603563

申请日：2015-01-23

Applicant: GoPro, Inc.

Inventor： Scott Patrick Campbell ,  David A. Newman ,  Brian G. Schunck

IPC: H04N5/335 ,  H04N5/341 ,  H04N19/426 ,  H04N19/423 ,  H04N5/232 ,  H04N1/21

CPC classification number: H04N1/2137 ,  H04N1/00209 ,  H04N5/23206 ,  H04N5/335 ,  H04N5/341 ,  H04N5/374 ,  H04N9/045 ,  H04N19/0049 ,  H04N19/137 ,  H04N19/156 ,  H04N19/172 ,  H04N19/423 ,  H04N19/426 ,  H04N19/625 ,  H04N19/63 ,  H04N19/91

Abstract: An image sensor compresses image data prior to transmitting the image data to a DSP. The image sensor captures light representing an image, for instance via a camera's aperture. A focal plane array converts the captured light into pixel data. The pixel data is sorted into categories, and is compressed in parallel by a compression engine. The compressed pixel data is then sent to a DSP, which may be located off-chip. The DSP then decompresses the compressed pixel data, performs image signal processing operations on the compressed pixel data, and then compresses the processed pixel data into a digital image format. The image sensor may buffer the pixel data for one or more images to accommodate for slowdown by the compression engine. The pixel data may be sorted by row and column of a pixel array. Alternatively, the pixel data may be sorted by color from a Bayer color filter.

公开(公告)号：US11800141B2

公开(公告)日：2023-10-24

申请号：US17453217

申请日：2021-11-02

Applicant: GoPro, Inc.

Inventor： David A. Newman ,  Cesar Douady ,  Alexis Lefebvre ,  Daryl Stimm ,  Michael Kraak

IPC: G06T3/00 ,  H04N19/40 ,  H04N19/52 ,  H04N19/182 ,  H04N19/196

CPC classification number: H04N19/52 ,  G06T3/0087 ,  H04N19/182 ,  H04N19/198 ,  H04N19/40

Abstract: Methods and apparatus for processing of video content to optimize codec bandwidth. In one embodiment, the method includes capturing panoramic imaging content (e.g., a 360° panorama), mapping the panoramic imaging content into an equi-angular cubemap (EAC) format, and splitting the EAC format into segments for transmission to maximize codec bandwidth. In one exemplary embodiment, the EAC segments are transmitted at a different frame rate than the subsequent display rate of the panoramic imaging content. For example, the mapping and frame rate may be chosen to enable the rendering of 8K, 360° content at 24 fps, using commodity encoder hardware and software that nominally supports 4K content at 60 fps.

公开(公告)号：US11762175B2

公开(公告)日：2023-09-19

申请号：US17079300

申请日：2020-10-23

Applicant: GoPro, Inc.

Inventor： David A. Newman

IPC: G02B13/16 ,  G02B3/08 ,  H04N23/698

CPC classification number: G02B13/16 ,  G02B3/08 ,  H04N23/698

Abstract: Methods and apparatus for seamlessly transitioning between lens projections. In one exemplary embodiment, a piecewise lens projection is composed of three (3) functions: (i) a first polynomial-based lens projection, (ii) a second “joining” lens projection, and (iii) a trigonometric lens projection. The piecewise lens projection characterizes virtualized lens distortion as a function of FOV; image data can be dynamically projected based on the virtualized lens distortion, regardless of FOV. In this manner, a user may achieve the visually familiar effects associated with a first lens definition for a first FOV, while still smoothly animating transitions to other lens projections (e.g., a larger FOV using stereographic projections).

公开(公告)号：US20230144635A1

公开(公告)日：2023-05-11

申请号：US18059827

申请日：2022-11-29

Applicant: GoPro, Inc.

Inventor： David A. Newman

IPC: H04N5/262 ,  H04N23/68 ,  G11B27/034

CPC classification number: H04N5/2621 ,  H04N23/6811 ,  G11B27/034

Abstract: Methods and apparatus for metadata-based cinematography, production effects, shot selection, and/or other content augmentation. Effective cinematography conveys storyline, emotion, excitement, etc. Unfortunately, most amateur filmmakers lack the knowledge and ability to create cinema quality media. Various aspects of the present disclosure are directed to, among other things, rendering media based on instantaneous metadata. Unlike traditional post-processing techniques that rely on human subjectivity, some of the various techniques described herein leverage the camera's actual experiential data to enable cinema-quality post-processing for the general consuming public. Instantaneous metadata-based cinematography and shot selection advisories and architectures are also described.

公开(公告)号：US11538138B2

公开(公告)日：2022-12-27

申请号：US17239586

申请日：2021-04-24

Applicant: GoPro, Inc.

Inventor： David A. Newman

IPC: G06T5/00 ,  G06T7/11 ,  H04N5/232 ,  G06T5/20 ,  G06T7/194

Abstract: Apparatus and methods for applying motion blur to overcapture content. In one embodiment, the motion blur is applied by selecting a number of frames of the captured image content for application of motion blur; selecting a plurality of pixel locations within the number of frames of the captured image content for the application of motion blur; applying motion blur to the captured image content in accordance with the selected number of frames and the selected plurality of pixel locations; and outputting the captured image content with the applied motion blur. In some implementations, motion blur is applied via implementation of a virtualized neutral density filter. Computerized devices and computer-readable apparatus for the application of motion blur are also disclosed.

公开(公告)号：US10817977B2

公开(公告)日：2020-10-27

申请号：US16884962

申请日：2020-05-27

Applicant: GoPro, Inc.

Inventor： David A. Newman ,  Joshua Edward Bodinet ,  Otto Kenneth Sievert ,  Timothy MacMillan

IPC: G06T3/00 ,  G06T5/00 ,  G06T3/40

Abstract: In a video capture system, a virtual lens is simulated when applying a crop or zoom effect to an input video. An input video frame is received from the input video that has a first field of view and an input lens distortion caused by a lens used to capture the input video frame. A selection of a sub-frame representing a portion of the input video frame is obtained that has a second field of view smaller than the first field of view. The sub-frame is processed to remap the input lens distortion to a desired lens distortion in the sub-frame. The processed sub-frame is the outputted.