Embodiments are generally directed to video analytics encoding for improved efficiency of video processing and compression. An embodiment of an apparatus includes a memory to store data, including data for video streaming, and a video processing mechanism, wherein the video processing mechanism is to analyze video data and generate video analytics, generate metadata representing the video analytics and insert the generated video analytics metadata into a message, and transmit the video data and the metadata to a succeeding apparatus or system in a video analytics pipeline, the video data being compressed video data.