A method for encoding a block of a video stream includes generating, using pixel values of the block, block features for the block; for each candidate encoding mode of candidate encoding modes, generating, using the block features and the each candidate encoding mode as inputs to a machine-learning module, a respective encoding cost; selecting, based on the respective encoding costs, a predetermined number of the candidate encoding modes; selecting, based on the respective encoding costs of the at least some encoding modes, a best mode for encoding the block; and encoding, in a compressed bitstream, the block using the best mode.