A hybrid video encoding method and system using a software engine and a hardware engine. The software engine receives coding unit data associated with a current picture, and performs a first part of the video encoding operation by executing instructions. The first part of the video encoding operation generates an inter predictor and control information corresponding to the coding unit data of the current picture. The first part of the video encoding operation stores the inter predictor into an off-chip memory. The hardware engine performs a second part of the video encoding operation according to the control information. The second part of the video encoding operation receives the inter predictor, and subtracts the inter predictor from the coding unit data to generate a residual signal. The second part of the video encoding operation then transforms and quantizes the residual signal to generate transformed and quantized residual signal, and encodes the transformed and quantized residual signal to generate an encoded video bitstream.