An analytics system uses one or more machine-learned models to predict a hormone receptor status from a H&E stain image. The system partitions H&E stain images each into a plurality of image tiles. Bags of tiles are created through sampling of the image tiles. The analytics system trains one or more machine-learned models with training H&E stain images having a positive or negative receptor status. The analytics system generates, via a tile featurization model, a tile feature vector for each image tile a test bag for a test H&E stain image. The analytics system generates, via an attention model, an aggregate feature vector for the test bag by aggregating the tile feature vectors of the test bag, wherein an attention weight is determined for each tile feature vector. The analytics system predicts a hormone receptor status by applying a prediction model to the aggregate feature vector for the test bag.