公开(公告)号：US20220308975A1

公开(公告)日：2022-09-29

申请号：US17717822

申请日：2022-04-11

Applicant: GOOGLE LLC

Inventor： Dragan Zivkovic ,  Harry Bleyan ,  Tamar Lucassen ,  Akash Agrawal

IPC: G06F11/34 ,  G06F8/65 ,  G06F11/36 ,  G06N20/00

Abstract: Implementations disclosed herein are directed to systems and methods for evaluating new feature(s) for client device(s) based on performance measure(s) of the client device(s) and/or the new feature(s). The new feature(s) can include, for example, machine learning (ML) model(s), non-ML software-enabled functionality, non-ML hardware-enabled functionality, and/or ML or non-ML software application features for a given software application utilized by the client device(s). The client device(s) can generate the performance measure(s) by processing a plurality of testing instances for the new feature(s). The performance measure(s) can include, for example, latency measure(s), memory consumption measure(s), CPU usage measure(s), precision and/or recall measure(s), and/or other measures. In some implementations, the new feature(s) may be activated for use locally at the client device(s) based on the performance measure(s), and optionally at other client device(s) that share the same device characteristics. In other implementations, the new feature(s) may be modified based on the performance measure(s).

公开(公告)号：US20220309389A1

公开(公告)日：2022-09-29

申请号：US17215588

申请日：2021-03-29

Applicant: Google LLC

Inventor： Dragan Zivkovic ,  Akash Agrawal ,  Françoise Beaufays ,  Tamar Lucassen

IPC: G06N20/00 ,  G06N5/04

Abstract: Implementations disclosed herein are directed to systems and methods for evaluating on-device machine learning (ML) model(s) based on performance measure(s) of client device(s) and/or the on-device ML model(s). The client device(s) can include on-device memory that stores the on-device ML model(s) and a plurality of testing instances for the on-device ML model(s). When certain condition(s) are satisfied, the client device(s) can process, using the on-device ML model(s), the plurality of testing instances to generate the performance measure(s). The performance measure(s) can include, for example, latency measure(s), memory consumption measure(s), CPU usage measure(s), ML model measure(s) (e.g., precision and/or recall), and/or other measures. In some implementations, the on-device ML model(s) can be activated (or kept active) for use locally at the client device(s) based on the performance measure(s). In other implementations, the on-device ML model(s) can be sparsified based on the performance measure(s).