post https://api-lib.bambu.life/api/autoMl/v2/ensemble
This endpoint is to train the data using auto-sklearn package.
-
The following estimators or classifiers are included: GaussianNB, BernoulliNB, Decision tree, Random forest, Extra tree, k nearest neighbors, Liblinear-svc, Passive-aggressive, LDA and QDA.
-
No preprocessors that predefined in auto-sklearn are used.
-
Main parameters:
a) timeLeftForThisTask: the total time for running this classification task (it can be set for a smaller number to estimate how long would a run lasts)
b) perRunTimeLimit: the limit for 1 run. It varies with respect to algorithms and sample size. It is recommended to estimate the time for each run by setting a small time-left-for-this-task.
-
Model can be saved for evaluation and prediction.
-
Output:
a) training: "statis": summary for training, "showEnsembleDetails" (optional): lists constructor and the corresponding weight in the ensemble
b) evaluation: accuracy, classification-report, multi_label confusion matrix
Request Body
| Name | Datatype | Description | Mandatory | Sample Value | List of possible values | Notes |
|---|---|---|---|---|---|---|
| clientId | String | It is meant for identification purpose as information are stored under unique id value | Yes | |||
| timeLeftForThisTask | Integer | Total training time is the time limit in seconds for the search of optimum combination of models that has the best performance evaluation . Increasing the time gives a higher chance of finding better models | Yes | 30 | >=30 | |
| perRunTimeLimit | Integer | It is time limit to train a single machine learning model to find the optimum tuning. Model fitting will be terminated if the machine learning algorithm runs over the time limit. It is good to set the time high enough so that the machine learning algorithm is able to fit on the training data and find optimum hyperparameter | Yes | 100 | >0 | |
| metric | String | Metric is used to measure and evaluate the performance of the machine learning algorithm. Different metric will give different performance result for trained model | Yes | "precision" | 'recall''precision''accuracy''f1Macro', 'f1Sample' | |
| evaluation | Boolean | If True, the output will contain validation results based on the selected metric as well as training resultsIf False, the output will only contain training results, such as model hyperparameter and training score | Yes | true | true or false | |
| showEnsembleDetails | Boolean | Show all the details of every model hyperparameters in ensemble | Yes | true | true or false | |
| saveModel | Boolean | Save model in database | Yes | true | true or false |
Response Body
| Name | Datatype | Description | Sample Value | No. of decimal places | Notes |
|---|---|---|---|---|---|
| classificationReport | Dictionary | A report containing performance measures of the algorithm. The dictionary contains 2 main keys: avgs and labels. avgs is an array of dictionaries that has keys of avgName and metrics. Valid avgName are: macro avg, micro avg, samples avg, weighted avg. Valid metrics are: precision, recall, f1-score, support. labels is an array of dictionaries that has keys of goalType value and metrics of precision, recall, f1-score, support | "classificationReport": { "labels": [ { "labelName": "retirement", "metrics": [ { "name": "precision", "value": 0.83051 }, { "name": "recall", "value": 0.85965 }, { "name": "f1-score", "value": 0.84483 }, { "name": "support", "value": 228 } ] }, { "labelName": "education", "metrics": [ { "name": "precision", "value": 0.76812 }, { "name": "recall", "value": 0.84574 }, { "name": "f1-score", "value": 0.80506 }, { "name": "support", "value": 188 } ] }, { "labelName": "house", "metrics": [ { "name": "precision", "value": 0.58025 }, { "name": "recall", "value": 0.5875 }, { "name": "f1-score", "value": 0.58385 }, { "name": "support", "value": 80 } ] }, { "labelName": "financialSecurity", "metrics": [ { "name": "precision", "value": 0.4 }, { "name": "recall", "value": 0.25 }, { "name": "f1-score", "value": 0.30769 }, { "name": "support", "value": 16 } ] }, { "labelName": "business", "metrics": [ { "name": "precision", "value": 0.48276 }, { "name": "recall", "value": 0.875 }, { "name": "f1-score", "value": 0.62222 }, { "name": "support", "value": 16 } ] } ], "avgs": [ { "avgName": "macro avg", "metrics": [ { "name": "precision", "value": 0.57916 }, { "name": "recall", "value": 0.59519 }, { "name": "f1-score", "value": 0.57291 }, { "name": "support", "value": 756 } ] }, { "avgName": "micro avg", "metrics": [ { "name": "precision", "value": 0.70038 }, { "name": "recall", "value": 0.7328 }, { "name": "f1-score", "value": 0.71622 }, { "name": "support", "value": 756 } ] }, { "avgName": "samples avg", "metrics": [ { "name": "precision", "value": 0.71111 }, { "name": "recall", "value": 0.74889 }, { "name": "f1-score", "value": 0.7104 }, { "name": "support", "value": 756 } ] }, { "avgName": "weighted avg", "metrics": [ { "name": "precision", "value": 0.70222 }, { "name": "recall", "value": 0.7328 }, { "name": "f1-score", "value": 0.71317 }, { "name": "support", "value": 756 } ] } ] }, | 5support: integer | |
| f1-score | Number | F1 Score is the weighted average of Precision and Recall. Therefore, this score takes both false positives and false negatives into account. F1 is usually more useful than accuracy, especially if you have an uneven class distribution. Accuracy works best if false positives and false negatives have similar cost. If the cost of false positives and false negatives are very different, it’s better to look at both Precision and Recall. F1 Score = 2(Recall Precision) / (Recall + Precision) | 5 | ||
| precision | Number | Precision is the ratio of correctly predicted positive observations to the total predicted positive observations. High precision relates to the low false positive rate. Precision = TP/TP+FP | 5 | ||
| recall | Number | Recall is the ratio of correctly predicted positive observations to all the positive observations in actual class. Recall = TP/TP + FN | 5 | ||
| support | Number | Support is the number of occurrences of each class in y_true | 5 | ||
| multilabelConfusionMatrix | Array of dictionary | It computes a confusion matrix for each class or sample (for this scenario, it is goalType) in the dataset. multilabelConfusionMatrix has keys of goalType , falseNegative, falsePositive, trueNegative and truePositive | "multilabelConfusionMatrix": [ { "falseNegative": 41, "falsePositive": 31, "name": "retirement", "trueNegative": 205, "truePositive": 23 }, { "falseNegative": 29, "falsePositive": 149, "name": "growWealth", "trueNegative": 63, "truePositive": 59 },... } | 0 | |
| numberOfRunningAlgorithms | Dictionary | It describes how many algorithms are successful and failure during training process | "numberOfRunningAlgorithms": { "total": 6, "success": 5, "crashed": 0, "exceededMemoryLimit": 0, "exceededTimeLimit": 1 }, | ||
| crashed | Integer | It describes how many crashed algorithms during training | 0 | 0 | |
| exceededMemoryLimit | Integer | It describes how many algorithms exceed the memory limit during training | 0 | 0 | |
| exceededTimeLimit | Integer | It describes how many algorithms exceed time limit during training | 1 | 0 | |
| success | Integer | It describes how many algorithms successfully trained | 5 | 0 | |
| total | Integer | It describes the total number of trained algorithms during training | 6 | 0 | |
| showEnsembleDetails | Array of dictionaries | It contains the list of machine learning models with their own hyperparameter details as well as the weight of the machine learning models on ensemble model. | "showEnsembleDetails": [ { "classifier": "random_forest_1", "weight": 0.4, "details": { "hyperParameters": [ { "name": "bootstrap", "value": "True" }, { "name": "criterion", "value": "gini" }, { "name": "max_depth", "value": "None" }, { "name": "max_features", "value": 0.5 }, { "name": "max_leaf_nodes", "value": "None" }, { "name": "min_impurity_decrease", "value": 0 }, { "name": "min_samples_leaf", "value": 1 }, { "name": "min_samples_split", "value": 2 }, { "name": "min_weight_fraction_leaf", "value": 0 }, { "name": "n_estimators", "value": 100 } ], "additionalInfo": [ { "name": "balancing:strategy", "value": "none" }, { "name": "categorical_encoding:choice", "value": "one_hot_encoding" }, { "name": "imputation:strategy", "value": "mean" }, { "name": "preprocessor:choice", "value": "no_preprocessing" }, { "name": "rescaling:choice", "value": "standardize" }, { "name": "categorical_encoding:one_hot_encoding:use_minimum_fraction", "value": "True" }, { "name": "categorical_encoding:one_hot_encoding:minimum_fraction", "value": 0.01 } ] } }, | ||
| statis | Dictionary | It contains the name of dataset, metric used | "statis": { "Dataset name": "57a8b04ef0bfa81e3ed3148c399b0bfa", "Metric": "precision_macro" } | ||
| trainingScore | Number | It describes the training score based on training dataset | 0.64257 | 5 | |
| evaluationScore | Number | It describes the validation score based on validation dataset | 0.59219 | 5 |