TECHNIQUES FOR TRAINING VISION FOUNDATION MODELS VIA MULTI-TEACHER DISTILLATION

    公开(公告)号:US20250165777A1

    公开(公告)日:2025-05-22

    申请号:US18740294

    申请日:2024-06-11

    Abstract: One embodiment of a method for training a first machine learning model includes processing first data via a plurality of trained machine learning models to generate a plurality of first outputs, processing the first data via the first machine learning model to generate a second output, processing the second output via a plurality of projection heads to generate a plurality of third outputs, computing a plurality of losses based on the plurality of first outputs and the plurality of third outputs, and performing one or more operations to update one or more parameters of the first machine learning model and one or more parameters of the plurality of projection heads based on the plurality of losses.

Patent Agency Ranking