公开(公告)号：US20170124486A1

公开(公告)日：2017-05-04

申请号：US15404052

申请日：2017-01-11

Applicant: salesforce.com, inc.

Inventor： Ka Hou Chan ,  Simon Chan ,  Kit Pang Szeto ,  Yue Kwen Justin Yip

IPC: G06N99/00 ,  G06N5/02

CPC classification number: G06N99/005 ,  G06F17/30477 ,  G06F17/30554 ,  G06N5/022 ,  G06N7/005 ,  G06Q30/0204

Abstract: In accordance with disclosed embodiments, there are provided systems, methods, and apparatuses for implementing predictive engine evaluation and replay of engine performance. An exemplary system may include, for example: means selecting a first set of one or more algorithms for a machine learning model; tuning a first group of predictive engine parameters for the machine learning model; training the machine learning model with one or more sources of data using the selected first set of one or more algorithms and the first group of tuned predictive engine parameters to generate a first predictive engine variant from the trained machine learning model; selecting a second set of one or more algorithms for a machine learning model which are different than the first set; tuning a second group of predictive engine parameters for the machine learning model which are different than the first group; training the machine learning model with the one or more sources of data using the selected second set of one or more algorithms and the second group of tuned predictive engine parameters to generate a second predictive engine variant from the trained machine learning model; performing multiple experiments using the first and second predictive engine variants; comparing results from the multiple experiments; and deploying either the first predictive engine variant or the second predictive engine variant based on the comparison of the results of the multiple experiments. Other related embodiments are disclosed.

公开(公告)号：US10192172B2

公开(公告)日：2019-01-29

申请号：US15404052

申请日：2017-01-11

Applicant: salesforce.com, inc.

Inventor： Ka Hou Chan ,  Simon Chan ,  Kit Pang Szeto ,  Yue Kwen Justin Yip

IPC: G06N99/00 ,  G06Q30/02 ,  G06F17/30 ,  G06N7/00 ,  G06N5/02

Abstract: In accordance with disclosed embodiments, there are provided systems, methods, and apparatuses for implementing predictive engine evaluation and replay of engine performance. An exemplary system may include, for example: means selecting a first set of one or more algorithms for a machine learning model; tuning a first group of predictive engine parameters for the machine learning model; training the machine learning model with one or more sources of data using the selected first set of one or more algorithms and the first group of tuned predictive engine parameters to generate a first predictive engine variant from the trained machine learning model; selecting a second set of one or more algorithms for a machine learning model which are different than the first set; tuning a second group of predictive engine parameters for the machine learning model which are different than the first group; training the machine learning model with the one or more sources of data using the selected second set of one or more algorithms and the second group of tuned predictive engine parameters to generate a second predictive engine variant from the trained machine learning model; performing multiple experiments using the first and second predictive engine variants; comparing results from the multiple experiments; and deploying either the first predictive engine variant or the second predictive engine variant based on the comparison of the results of the multiple experiments. Other related embodiments are disclosed.

公开(公告)号：US20160275406A1

公开(公告)日：2016-09-22

申请号：US14997662

申请日：2016-01-18

Applicant: salesforce.com, inc.

Inventor： Ka Hou Chan ,  Simon Chan ,  Kit Pang Szeto ,  Yue Kwen Justin Yip

IPC: G06N7/00 ,  G06F17/30

CPC classification number: G06N20/00 ,  G06F16/2455 ,  G06F16/248 ,  G06N5/022 ,  G06N7/005 ,  G06Q30/0204

Abstract: Disclosed are methods and systems of tracking the deployment of a predictive engine for machine learning, including steps to deploy an engine variant of the predictive engine based on an engine parameter set, wherein the engine parameter set identifies at least one data source and at least one algorithm; receive one or more queries to the deployed engine variant from one or more end-user devices, and in response, generate predicted results; receive one or more actual results corresponding to the predicted results; associate the queries, the predicted results, and the actual results with a replay tag, and record them with the corresponding deployed engine variant.

Abstract translation: 公开了跟踪用于机器学习的预测引擎的部署的方法和系统，包括基于引擎参数集部署预测引擎的引擎变体的步骤，其中引擎参数集识别至少一个数据源和至少一个 算法; 从一个或多个最终用户设备接收对所部署的引擎变体的一个或多个查询，作为响应，生成预测结果; 接收与预测结果相对应的一个或多个实际结果; 使用重播标签关联查询，预测结果和实际结果，并使用相应的部署引擎变体进行记录。