公开(公告)号：US11451710B2

公开(公告)日：2022-09-20

申请号：US16935476

申请日：2020-07-22

Applicant: Microsoft Technology Licensing, LLC

Inventor： Matthew Wozniak ,  Bakkama Srinath Reddy ,  Amr Martini

IPC: H04N5/232 ,  H04N19/184 ,  H04N19/33 ,  G06K9/00 ,  G06T11/00 ,  G06V20/40

Abstract: Innovations in stream configuration operations and playback operations for adaptive streaming of panoramic video are described. The innovations include features of adaptive streaming of panoramic video with composite pictures. For example, a stream configuration tool splits an input picture of panoramic video into multiple sections and creates multiple composite pictures. A composite picture includes one of the sections as well as a low-resolution version of the input picture. A playback tool reconstructs one or more composite pictures. Under normal operation, the playback tool can use the reconstructed section(s) of the composite picture(s) to render high-quality views of the panoramic video. If the view window dramatically changes, however, or if encoded data for a section is lost or corrupted, the playback tool can use the low-resolution version of the input picture to render lower-quality details for views of the panoramic video, without disruption of playback.

公开(公告)号：US20200351442A1

公开(公告)日：2020-11-05

申请号：US16935476

申请日：2020-07-22

Applicant: Microsoft Technology Licensing, LLC

Inventor： Matthew Wozniak ,  Bakkama Srinath Reddy ,  Amr Martini

IPC: H04N5/232 ,  H04N19/184 ,  H04N19/33 ,  G06K9/00 ,  G06T11/00

Abstract: Innovations in stream configuration operations and playback operations for adaptive streaming of panoramic video are described. The innovations include features of adaptive streaming of panoramic video with composite pictures. For example, a stream configuration tool splits an input picture of panoramic video into multiple sections and creates multiple composite pictures. A composite picture includes one of the sections as well as a low-resolution version of the input picture. A playback tool reconstructs one or more composite pictures. Under normal operation, the playback tool can use the reconstructed section(s) of the composite picture(s) to render high-quality views of the panoramic video. If the view window dramatically changes, however, or if encoded data for a section is lost or corrupted, the playback tool can use the low-resolution version of the input picture to render lower-quality details for views of the panoramic video, without disruption of playback.

公开(公告)号：US10764494B2

公开(公告)日：2020-09-01

申请号：US15990548

申请日：2018-05-25

Applicant: Microsoft Technology Licensing, LLC

Inventor： Matthew Wozniak ,  Bakkama Srinath Reddy ,  Amr Martini

IPC: H04N19/33 ,  H04N5/232 ,  H04N19/184 ,  G06K9/00 ,  G06T11/00

Abstract: Innovations in stream configuration operations and playback operations for adaptive streaming of panoramic video are described. The innovations include features of adaptive streaming of panoramic video with composite pictures. For example, a stream configuration tool splits an input picture of panoramic video into multiple sections and creates multiple composite pictures. A composite picture includes one of the sections as well as a low-resolution version of the input picture. A playback tool reconstructs one or more composite pictures. Under normal operation, the playback tool can use the reconstructed section(s) of the composite picture(s) to render high-quality views of the panoramic video. If the view window dramatically changes, however, or if encoded data for a section is lost or corrupted, the playback tool can use the low-resolution version of the input picture to render lower-quality details for views of the panoramic video, without disruption of playback.

公开(公告)号：US20190246084A1

公开(公告)日：2019-08-08

申请号：US16269230

申请日：2019-02-06

Applicant: Microsoft Technology Licensing, LLC

Inventor： Matthew Wozniak ,  Bakkama Srinath Reddy ,  Vladislav Alexandrov ,  David H. Mebane ,  Dibyajyoti Roy ,  Shyam Sadhwani

IPC: H04N9/31 ,  H04N19/597 ,  H04N19/85 ,  H04N19/44 ,  G06T3/00 ,  H04N9/64

Abstract: Innovations in reconstruction and rendering of panoramic video are described. For example, a view-dependent operation controller of a panoramic video playback system receives an indication of a view direction for an application and, based at least in part on the view direction, identifies a section of a picture of panoramic video in an input projection. The view-dependent operation controller limits operations of a color converter, video decoder, and/or streaming controller to the identified section. In this way, the panoramic video playback system can avoid performing operations to reconstruct sections of the picture of panoramic video that will not be viewed. As another example, a mapper of a panoramic video playback system re-projects at least some sample values in an input flat projection towards a center location for a view direction, producing an output flat projection, which an application can use to generate one or more screen projections.

公开(公告)号：US10271069B2

公开(公告)日：2019-04-23

申请号：US15253526

申请日：2016-08-31

Applicant: Microsoft Technology Licensing, LLC

Inventor： Bakkama Srinath Reddy ,  Shyam Sadhwani ,  Victor Cherepanov

IPC: H04N19/70 ,  H04N19/169 ,  H04N21/44

Abstract: Approaches to selectively using start code emulation prevention (“SCEP”) on encoded data for media content are described herein. For example, a media encoder selectively performs SCEP processing on encoded data for media content, and sets a value of a syntax element that indicates whether or not to perform SCEP processing on the encoded data. The encoder stores the encoded data for output as part of a bitstream, where the syntax element is signaled in association with the bitstream. A media decoder receives the encoded data, determines, from the value of the syntax element, whether or not to perform SCEP processing on the encoded data, and selectively performs SCEP processing on the encoded data. In this way, the computational cost of scanning operations for SCEP processing can be avoided in many scenarios, and bit rate increases due to insertion of SCPE bytes can be limited.

公开(公告)号：US20180152663A1

公开(公告)日：2018-05-31

申请号：US15384237

申请日：2016-12-19

Applicant: Microsoft Technology Licensing, LLC

Inventor： Matthew Wozniak ,  Bakkama Srinath Reddy ,  Vladislav Alexandrov ,  David H. Mebane ,  Dibyajyoti Roy ,  Shyam Sadhwani

IPC: H04N5/775 ,  H04N5/232 ,  G06T3/20 ,  H04N19/44 ,  G06T7/90 ,  H04N19/136 ,  G06T7/13

CPC classification number: H04N5/775 ,  G06T3/0037 ,  G06T3/0043 ,  G06T3/005 ,  G06T3/0062 ,  G06T3/20 ,  G06T7/13 ,  G06T7/90 ,  H04N5/23238 ,  H04N5/76 ,  H04N9/67 ,  H04N13/243 ,  H04N19/136 ,  H04N19/44

Abstract: Innovations in reconstruction and rendering of panoramic video are described. For example, a view-dependent operation controller of a panoramic video playback system receives an indication of a view direction for an application and, based at least in part on the view direction, identifies a section of a picture of panoramic video in an input projection. The view-dependent operation controller limits operations of a color converter, video decoder, and/or streaming controller to the identified section. In this way, the panoramic video playback system can avoid performing operations to reconstruct sections of the picture of panoramic video that will not be viewed. As another example, a mapper of a panoramic video playback system re-projects at least some sample values in an input flat projection towards a center location for a view direction, producing an output flat projection, which an application can use to generate one or more screen projections.

公开(公告)号：US20170127074A1

公开(公告)日：2017-05-04

申请号：US14927127

申请日：2015-10-29

Applicant: Microsoft Technology Licensing, LLC

Inventor： Yongjun Wu ,  Shyam Sadhwani ,  Bakkama Srinath Reddy

IPC: H04N19/44 ,  G06T3/40 ,  H04N19/70 ,  H04N19/136 ,  H04N19/176 ,  H04N19/423

CPC classification number: H04N19/44 ,  H04N19/423

Abstract: Aspects extend to methods, systems, and computer program products for video bit stream decoding. Aspects include flexible definition and detection of surface alignment requirements for decoding hardware. Surface alignment requirements can be handled by render cropping (e.g., cropping at a video output device), through adjustment and modification of original syntax values in a video bit stream and relaxed media type negotiation in a software (host) decoder. Resolution changes can be hidden with the aligned surface allocation when applicable. Performance can be improved and power consumption reduced by using hidden resolution changes.

公开(公告)号：US10750205B2

公开(公告)日：2020-08-18

申请号：US16294680

申请日：2019-03-06

Applicant: Microsoft Technology Licensing, LLC

Inventor： Bakkama Srinath Reddy ,  Shyam Sadhwani ,  Victor Cherepanov

IPC: H04N19/70 ,  H04N21/44

Abstract: Approaches to selectively using start code emulation prevention (“SCEP”) on encoded data for media content are described herein. For example, a media encoder selectively performs SCEP processing on encoded data for media content, and sets a value of a syntax element that indicates whether or not to perform SCEP processing on the encoded data. The encoder stores the encoded data for output as part of a bitstream, where the syntax element is signaled in association with the bitstream. A media decoder receives the encoded data, determines, from the value of the syntax element, whether or not to perform SCEP processing on the encoded data, and selectively performs SCEP processing on the encoded data. In this way, the computational cost of scanning operations for SCEP processing can be avoided in many scenarios, and bit rate increases due to insertion of SCEP bytes can be limited.

公开(公告)号：US10735725B2

公开(公告)日：2020-08-04

申请号：US15265651

申请日：2016-09-14

Applicant: Microsoft Technology Licensing, LLC

Inventor： Bakkama Srinath Reddy ,  Victor Cherepanov

IPC: H04N19/117 ,  H04N19/86 ,  H04N19/82 ,  H04N19/139 ,  H04N19/176

Abstract: A video encoder or video decoder buffers multiple blocks of a reconstructed picture of a video sequence. The video encoder/decoder performs deblock filtering between at least some of the multiple blocks. As part of the deblock filtering, the video encoder/decoder selectively filters at least some sample values in a diagonal line that crosses a block-boundary intersection between two diagonally adjacent blocks. When filtering sample values at the block-boundary intersection between four blocks, the video encoder/decoder can evaluate characteristics of all four blocks and adjust sample values in a line between diagonally adjacent blocks. If there is a large visual difference between sample values at corner positions of two diagonally adjacent blocks, the difference can be smoothed by filtering sample values in a diagonal line. In this way, the quality of motion-compensated prediction using the reconstructed picture is improved in many cases.

公开(公告)号：US10666863B2

公开(公告)日：2020-05-26

申请号：US15990557

申请日：2018-05-25

Applicant: Microsoft Technology Licensing, LLC

Inventor： Matthew Wozniak ,  Bakkama Srinath Reddy ,  Amr Martini

IPC: H04N5/232 ,  H04N19/184 ,  H04N19/30 ,  G06K9/00 ,  G06T11/00

Abstract: Innovations in stream configuration operations and playback operations for adaptive streaming of panoramic video are described. The innovations include features of adaptive streaming of panoramic video with overlapping sections. For example, a stream configuration tool splits an input picture of panoramic video into overlapping sections. A playback tool reconstructs one or more of the sections. Because of overlap between adjacent sections, for gradual changes in a view window, the playback tool can, in effect, preemptively fetch a new section as the view window moves out of a current section into the new section. By the time the view window reaches a non-overlapping part of the adjacent, new section, the playback tool has retrieved encoded data for the new section and can reconstruct the new section. This hides network latency and stream switching latency from the viewer, and disruption of playback is avoided.