公开(公告)号：US11637971B2

公开(公告)日：2023-04-25

申请号：US17401895

申请日：2021-08-13

Applicant: GoPro, Inc.

Inventor： Balineedu Chowdary Adsumilli ,  Ryan Lustig

IPC: H04N5/272 ,  G06T15/50 ,  H04N13/128 ,  G06T11/60 ,  G06T7/11 ,  G06T7/143 ,  G06T7/194 ,  G06T7/174 ,  G06T7/254 ,  G06T5/00 ,  G06T7/20 ,  H04N5/235 ,  H04N13/00

Abstract: A processing device generates composite images from a sequence of images. The composite images may be used as frames of video. A foreground/background segmentation is performed at selected frames to extract a plurality of foreground object images depicting a foreground object at different locations as it moves across a scene. The foreground object images are stored to a foreground object list. The foreground object images in the foreground object list are overlaid onto subsequent video frames that follow the respective frames from which they were extracted, thereby generating a composite video.

公开(公告)号：US20210377460A1

公开(公告)日：2021-12-02

申请号：US17401895

申请日：2021-08-13

Applicant: GoPro, Inc.

Inventor： Balineedu Chowdary Adsumilli ,  Ryan Lustig

IPC: H04N5/272 ,  G06T15/50 ,  H04N13/128 ,  G06T11/60 ,  G06T7/11 ,  G06T7/143 ,  G06T7/194 ,  G06T7/174 ,  G06T7/254 ,  G06T5/00 ,  G06T7/20 ,  H04N5/235

Abstract: A processing device generates composite images from a sequence of images. The composite images may be used as frames of video. A foreground/background segmentation is performed at selected frames to extract a plurality of foreground object images depicting a foreground object at different locations as it moves across a scene. The foreground object images are stored to a foreground object list. The foreground object images in the foreground object list are overlaid onto subsequent video frames that follow the respective frames from which they were extracted, thereby generating a composite video.

公开(公告)号：US10192585B1

公开(公告)日：2019-01-29

申请号：US15467804

申请日：2017-03-23

Applicant: GOPRO, INC.

Inventor： Balineedu Chowdary Adsumilli ,  Ryan Lustig ,  David A. Newman

IPC: G11B27/10 ,  G06K9/00 ,  G10L15/22 ,  G10L15/06 ,  G11B27/031 ,  H04N5/77 ,  H04N5/91

Abstract: Video and corresponding metadata is accessed. Events of interest within the video are identified based on the corresponding metadata, and best scenes are identified based on the identified events of interest. In one example, best scenes are identified based on the motion values associated with frames or portions of a frame of a video. Motion values are determined for each frame and portions of the video including frames with the most motion are identified as best scenes. Best scenes may also be identified based on the motion profile of a video. The motion profile of a video is a measure of global or local motion within frames throughout the video. For example, best scenes are identified from portion of the video including steady global motion. A video summary can be generated including one or more of the identified best scenes.

公开(公告)号：US10134114B2

公开(公告)日：2018-11-20

申请号：US15270971

申请日：2016-09-20

Applicant: GoPro, Inc.

Inventor： Balineedu Chowdary Adsumilli ,  Ryan Lustig ,  Aaron Staranowicz

IPC: G06K9/00 ,  G06T5/00 ,  G06T3/40

Abstract: Apparatus and methods for video image post-processing for segmentation based interpolation. In one embodiment, a computerized apparatus is utilized in order to obtain a first frame of video data; segment one or more objects within the first frame of video data; obtain a second frame of video data; segment one or more objects within the second frame of video data; match at least a portion of the one or more objects within the first frame of video data with the one or more objects within the second frame of video data; compute the motion of the pixels for the matched portion of the one or more objects; compute the motion of pixels associated with a background image; and generate an interpolated frame of video data, the interpolated frame of video data residing temporally between the first frame of video data and the second frame of video data.

公开(公告)号：US09965703B2

公开(公告)日：2018-05-08

申请号：US15177310

申请日：2016-06-08

Applicant: GoPro, Inc.

Inventor： Balineedu Chowdary Adsumilli ,  Ryan Lustig

IPC: G06K9/62 ,  G06N99/00 ,  G06K9/00

CPC classification number: G06K9/6262 ,  G06K9/00228 ,  G06K9/00744 ,  G06K9/6227 ,  G06K9/6229 ,  G06K9/6256 ,  G06N3/126 ,  G06N99/005

Abstract: An algorithm for performing an image or video processing task is generated that may be used to combine a plurality of different independent solutions to the image or video processing task in an optimized manner. A plurality of base algorithms may be applied to a training set of images or video and a first generation of different combining algorithms may be applied to combine the respective solutions from each of the respective base algorithms into respective combined solutions. The respective combined solutions may be evaluated to generate respective fitness scores representing measures of how well the plurality of different combining algorithms each perform the image or video processing task. The algorithms may be iteratively updated to generate an optimized combining algorithm that may be applied to an input image or video.

公开(公告)号：US09930271B2

公开(公告)日：2018-03-27

申请号：US15080280

申请日：2016-03-24

Applicant: GoPro, Inc.

Inventor： Balineedu Chowdary Adsumilli ,  Ryan Lustig

IPC: G06K9/00 ,  H04N5/272 ,  G06T15/50 ,  G06T5/00 ,  G06T7/20 ,  H04N5/235 ,  H04N13/00 ,  G06T11/60 ,  G06T7/11 ,  G06T7/143 ,  G06T7/194 ,  G06T7/174 ,  G06T7/254

CPC classification number: H04N5/272 ,  G06T5/002 ,  G06T7/11 ,  G06T7/143 ,  G06T7/174 ,  G06T7/194 ,  G06T7/20 ,  G06T7/254 ,  G06T11/60 ,  G06T15/503 ,  G06T2207/10016 ,  G06T2207/10021 ,  G06T2207/20036 ,  H04N5/2356 ,  H04N13/128 ,  H04N2013/0085 ,  H04N2013/0092

Abstract: A processing device generates composite images from a sequence of images. The composite images may be used as frames of video. A foreground/background segmentation is performed at selected frames to extract a plurality of foreground object images depicting a foreground object at different locations as it moves across a scene. The foreground object images are stored to a foreground object list. The foreground object images in the foreground object list are overlaid onto subsequent video frames that follow the respective frames from which they were extracted, thereby generating a composite video.

公开(公告)号：US20180082404A1

公开(公告)日：2018-03-22

申请号：US15270971

申请日：2016-09-20

Applicant: GoPro, Inc.

Inventor： Balineedu Chowdary Adsumilli ,  Ryan Lustig ,  Aaron Staranowicz

IPC: G06T5/00 ,  G06T3/40

CPC classification number: G06T5/002 ,  G06T3/4007 ,  G06T5/003 ,  G06T7/248 ,  G06T2207/10016 ,  G06T2207/10024

Abstract: Apparatus and methods for video image post-processing for segmentation based interpolation. In one embodiment, a computerized apparatus is utilized in order to obtain a first frame of video data; segment one or more objects within the first frame of video data; obtain a second frame of video data; segment one or more objects within the second frame of video data; match at least a portion of the one or more objects within the first frame of video data with the one or more objects within the second frame of video data; compute the motion of the pixels for the matched portion of the one or more objects; compute the motion of pixels associated with a background image; and generate an interpolated frame of video data, the interpolated frame of video data residing temporally between the first frame of video data and the second frame of video data.

公开(公告)号：US20170366755A1

公开(公告)日：2017-12-21

申请号：US15249347

申请日：2016-08-26

Applicant: GoPro, Inc.

Inventor： Ryan Lustig ,  Balineedu Chowdary Adsumilli

IPC: H04N5/232

CPC classification number: H04N5/23258 ,  H04N5/23267 ,  H04N5/2328

Abstract: A target image captured from a fisheye lens or other lens with known distortion parameters may be transformed to align it to a reference image. Corresponding features may be detected in the target image and the reference image. The features may be transformed to a spherical coordinate space. In the spherical space, images may be re-pointed or rotated in three dimensions to align all or a subset of the features of the target image to the corresponding features of the reference image. For example, in a sequence of images, background features of the target image in the spherical image space may be aligned to background features of the reference image in the spherical image space to compensate for camera motion while preserving foreground motion. An inverse transformation may then be applied to bring the images back into the original image space.

公开(公告)号：US20170357880A1

公开(公告)日：2017-12-14

申请号：US15177310

申请日：2016-06-08

Applicant: GoPro, Inc.

Inventor： Balineedu Chowdary Adsumilli ,  Ryan Lustig

IPC: G06K9/62 ,  G06N99/00 ,  G06K9/00

CPC classification number: G06K9/6262 ,  G06K9/00228 ,  G06K9/00744 ,  G06K9/6227 ,  G06K9/6229 ,  G06K9/6256 ,  G06N3/126 ,  G06N99/005

Abstract: An algorithm for performing an image or video processing task is generated that may be used to combine a plurality of different independent solutions to the image or video processing task in an optimized manner. A plurality of base algorithms may be applied to a training set of images or video and a first generation of different combining algorithms may be applied to combine the respective solutions from each of the respective base algorithms into respective combined solutions. The respective combined solutions may be evaluated to generate respective fitness scores representing measures of how well the plurality of different combining algorithms each perform the image or video processing task. The algorithms may be iteratively updated to generate an optimized combining algorithm that may be applied to an input image or video.

公开(公告)号：US09666232B2

公开(公告)日：2017-05-30

申请号：US15091539

申请日：2016-04-05

Applicant: GOPRO, INC.

Inventor： Balineedu Chowdary Adsumilli ,  Ryan Lustig ,  David A. Newman

IPC: H04N9/80 ,  H04N5/93 ,  G11B27/10 ,  G06K9/00 ,  G11B27/30 ,  G11B27/031 ,  G10L15/06 ,  G10L15/22 ,  H04N5/77 ,  H04N5/91 ,  H04N9/82 ,  G11B27/13 ,  G11B27/34 ,  G10L25/57 ,  G10L25/54

CPC classification number: G11B27/10 ,  G06K9/00718 ,  G06K9/00751 ,  G06K2009/00738 ,  G10L15/063 ,  G10L15/22 ,  G10L25/54 ,  G10L25/57 ,  G10L2015/0631 ,  G10L2015/223 ,  G11B27/031 ,  G11B27/13 ,  G11B27/28 ,  G11B27/3081 ,  G11B27/34 ,  H04N5/77 ,  H04N5/772 ,  H04N5/91 ,  H04N9/8205

Abstract: Video and corresponding metadata is accessed. Events of interest within the video are identified based on the corresponding metadata, and best scenes are identified based on the identified events of interest. In one example, best scenes are identified based on the motion values associated with frames or portions of a frame of a video. Motion values are determined for each frame and portions of the video including frames with the most motion are identified as best scenes. Best scenes may also be identified based on the motion profile of a video. The motion profile of a video is a measure of global or local motion within frames throughout the video. For example, best scenes are identified from portion of the video including steady global motion. A video summary can be generated including one or more of the identified best scenes.