Abstract:
A method and system of using a pre-encoder to improve encoder efficiency. The encoder may conform to ITU-T H.265 and the pre-encoder may conform to ITU-T H. 264. The pre-encoder may receive source video data and provide information regarding various coding modes, candidate modes, and a selected mode for coding the source video data. In an embodiment, the encoder may directly use the mode selected by the pre-encoder. In another embodiment, the encoder may receive both the source video data and information regarding the various coding modes (e.g., motion information, macroblock size, intra prediction direction, rate-distortion cost, and block pixel statistics) to simplify and/or refine its mode decision process. For example, the information provided by the pre-encoder may indicate unlikely modes, which unlikely modes need not be tested by the encoder, thus saving power and time.
Abstract:
Systems and methods are provided for capturing high quality video data, including data having a high dynamic range, for use with conventional encoders and decoders. High dynamic range data is captured using multiple groups of pixels where each group is captured using different exposure times to create groups of pixels. The pixels that are captured at different exposure times may be determined adaptively based on the content of the image, the parameters of the encoding system, or on the available resources within the encoding system. The transition from single exposure to using two different exposure times may be implemented gradually.
Abstract:
Video object and keypoint location detection techniques are presented. The system includes a detection system for generation locations of an object's keypoints along with probabilities associated with the locations, and a stability system for stabilizing keypoint locations of the detected objects. In some aspects, the generated probabilities are two-dimensional array correspond locations within input images, and stability system fits the generated probabilities to a two-dimensional probability distribution function.
Abstract:
Embodiments of the present disclosure provide systems and methods for perspective shifting in a video conferencing session. In one exemplary method, a video stream may be generated. A foreground element may be identified in a frame of the video stream and distinguished from a background element of the frame. Data may be received representing a viewing condition at a terminal that will display the generated video stream. The frame of the video stream may be modified based on the received data to shift of the foreground element relative to the background element. The modified video stream may be displayed at the displaying terminal.
Abstract:
In an example method, a system accesses first input data and a machine learning architecture. The machine learning architecture includes a first module having a first neural network, a second module having a second neural network, and a third module having a third neural network. The system generates a first feature set representing a first portion of the first input data using the first neural network, and a second feature set representing a second portion of the first input data using the second neural network. The system generates, using the third neural network, first output data based on the first feature set and the second feature set.
Abstract:
Methods and Systems disclosed to counteract spatial distortions introduced by imaging processes of multi-directional video frames, where objects may be projected to spherical or equirectangular representations. Techniques provided to invert the spatial distortions in video frames used as reference picture data in predictive coding, by spatially transforming the image content of the reference picture data before this image content is being used for the prediction of input video data in prediction-based coders and decoders.
Abstract:
Techniques are disclosed for coding video data predictively based on predictions made from spherical-domain projections of input pictures to be coded and reference pictures that are prediction candidates. Spherical projection of an input picture and the candidate reference pictures may be generated. Thereafter, a search may be conducted for a match between the spherical-domain representation of a pixel block to be coded and a spherical-domain representation of the reference picture. On a match, an offset may be determined between the spherical-domain representation of the pixel block to a matching portion of the of the reference picture in the spherical-domain representation. The spherical-domain offset may be transformed to a motion vector in a source-domain representation of the input picture, and the pixel block may be coded predictively with reference to a source-domain representation of the matching portion of the reference picture.
Abstract:
Frame packing techniques are disclosed for multi-directional images and video. According to an embodiment, a multi-directional source image is reformatted into a format in which image data from opposing fields of view are represented in respective regions of the packed image as flat image content. Image data from a multi-directional field of view of the source image between the opposing fields of view are represented in another region of the packed image as equirectangular image content. It is expected that use of the formatted frame will lead to coding efficiencies when the formatted image is processed by predictive video coding techniques and the like.
Abstract:
Judder artifacts are remedied in video coding system by employing frame rate conversion at an encoder. A source video sequence may be coded as base layer coded video at a first frame rate. An encoder may identify a portion of the coded video sequence that likely will exhibit judder effects when decoded. For those portions that likely will exhibit judder effects, video data representing the portion of the source video may be coded at a higher frame rate than a frame rate of the coded base layer data as enhancement layer data. Moreover, an encoder may generate metadata representing “FRC hints”—techniques that a decoder should employ when performing decoder-side frame rate conversion. An encoding terminal may transmit the base layer coded video and either the enhancement layer coded video or the FRC hints to a decoder. Thus, encoder infrastructure may mitigate against judder artifacts that may arise during decoding.
Abstract:
Techniques are disclosed for correcting artifacts in multi-view images that include a plurality of planar views. Image content the planar views may be projected from the planar representation to a spherical projection. Thereafter, a portion of the image content may be projected from the spherical projection to a planar representation. The image content of the planar representation may be used for display. Extensions are disclosed that correct artifacts that may arise during deblocking filtering of the multi-view images.