In some examples, a layered encoding component and a layered decoding component provide for different ways to encode and decode, respectively, video streams transmitted between devices. For instance, in encoding a video stream, a layered encoding component may analyze the content of successive video frames and determine different types of encoding techniques to use for different ones of the video frames. Further, in some cases, some of the encoding techniques may be used on less than an entire video frame. In another example, in decoding a video stream, a layered decoding component may receive video frames encoded with different types of encoding. The layered decoding component may decode the differently encoded video frames and combine them to reconstruct a video stream.