2.3.21 ModelResultSet
Model objects from a pipeline come back from the server organized into ModelResultSets and Configurations. A ModelResultSet represents everything returned by the pipeline and a Configuration is a subset pertaining to a single Train-Validate-Optimize (TVO) configuration or combination. It is common for a ModelResultSet to have a configuration containing many models. See the tutorials for more information about models, their metrics and recognition capabilities.
- class mplabml.datamanager.modelresults. ModelMetrics(configuration, sandbox, index, model_result)
Base class for a model metrics object.
- confusion_matrix_stats
Comprehensive metrics returned for the model
- Type
list[ConfusionMatrix]
- train_set
Indices of the input data that the model was trained with
- Type
list
- validation_set
Indices of the input data that the model was validated with
- Type
list
- test_set
Indices of the input data that the model was tested with
- Type
list
- debug
Structure containing debug information for some models
- Type
dict
- neurons
Model neuron array
- Type
list[dict]
- parameters
Model parameters
- Type
dict
- knowledgepack
Knowledgepack associated with the model
- Type
- property knowledgepack
The model’s KnowledgePack object
- recognize_signal(capturefile=None, stop_step=False, datafile=None, segmenter=True, lock=True, silent=True, platform='emulator', compare_labels=False, **kwargs)
Sends a DataFrame of raw signals to be run through the feature generation pipeline and recognized
- Parameters
capturefile (str) – The name of a file uploaded through the data capture lab
datafile (str) – The name of an uploading datafile
platform (str) – Emulator or cloud. The emulator will run compiled C code, giving device exact results, the cloud runs similarly to training providing more flexibility in returning early results by setting the stop step.
stop_step (int) – For debugging, if you want to stop the pipeline at a particular step, set stop_step to its index
compare_labels (bool) – If there are labels for the input DataFrame, use them to create a confusion matrix
segmenter (bool or FunctionCall) – To suppress or override the segmentation algorithm in the original pipeline, set this to False or a function call of type segmenter (defaults to True)
lock (bool , True) – If True, waits for the result to return before releasing the ipython cell
- Returns
- A DataFrame containing the results of recognition and a dictionary containing the execution summary and the confusion_matrix when labels are provided.
execution_summary: A summary of steps run in the execution engine
confusion_matrix: A confusion matrix, only if the input data has a Label column
- Return type
(DataFrame, dict)
- recognize_vectors(vectors)
Sends a DataFrame of feature vectors to the model’s KnowledgePack for recognition
- Parameters
vectors (DataFrame) – Where each row is a feature vector with column headings named the same as
features generated by the pipeline (the) –
label columns may be included (and) –
- Returns
Contains the results of recognition, including predicted class, neuron ID and distance
- Return type
(DataFrame)
- summarize(metrics_set='validation')
Prints a formatted summary of the model metrics
- Parameters
metrics_set (optional [ str ]) – Options are validation (the default), test and train
- class mplabml.datamanager.modelresults. ModelResultSet(project, sandbox)
Base class for a model result set object
- get_knowledgepack_by_index(config_index, model_index)
Returns the KnowledgePack of the given configuration index and model index
- sort_models(metric='accuracy', metrics_set='validation')
Sorts the models within all configurations by a given metric or property
- Parameters
metric (optional [ str ]) – Options are accuracy (the default), index and number_of_neurons
metrics_set (optional [ str ]) – Options are validation (the default), test and train
- summarize(metrics_set='validation', report_number=5)
Prints a basic summary of each configuration and its models
- Parameters
metrics_set (optional [ str ]) – Options are validation (the default), test and train
- class mplabml.datamanager.confusion_matrix. ConfusionMatrix(metrics_dict)
This object is a representation of a confusion matrix that contains properties for statistics that can be generated from a confusion matrix as well as a DataFrame representation for easy viewing
- pp_by_class_name(class_name)
Returns the positive predictivity of a given class in the confusion matrix
- Parameters
class_name (str) – Name of the class to return positive predictivity of
- Returns
If a class name exists in matrix return PP, else NaN
- Return type
pp_by_class
- se_by_class_name(class_name)
Returns the sensitivity of a given class in the confusion matrix
- Parameters
class_name (str) – Name of the class to return sensitivity of
- Returns
If class name exists in matrix return SE, else NaN
- Return type
se_by_class