eval_classifier¶

In this module, classes and methods are defined for evluating the performance of the TwinSVM model. Also, a method for saving detailed classification result.

Functions

`eval_metrics`(y_true, y_pred)	It computes common evaluation metrics such as Accuracy, Recall, Precision, F1-measure, and elements of the confusion matrix.
`grid_search`(search_space, func_validator)	It does grid search to find the optimcal values of hyperparameters for the TwinSVM model, which results in the best classfication accuracy.
`initializer`(user_input_obj)	It passes a user’s input to the functions and classes for solving a classification task.
`save_result`(file_name, validator_obj, …)	It saves the detailed classification results in a spreadsheet file (Excel).
`search_space`(kernel_type, class_type, …[, …])	It generates all combination of search elements based on the given range of hyperparameters.

Classes

Validator(X_train, y_train, validator_type, …) It evaluates the TwinSVM model based on the specified evaluation method.

eval_classifier.eval_metrics(y_true, y_pred)[source]¶

It computes common evaluation metrics such as Accuracy, Recall, Precision, F1-measure, and elements of the confusion matrix.

Parameters:

y_true : array-like

Target values of samples.

y_pred : array-like

Predicted class lables.

Returns:

tp : int

True positive.

tn : int

True negative.

fp : int

False positive.

fn : int

False negative.

accuracy : float

Overall accuracy of the model.

recall_p : float

Recall of positive class.

precision_p : float

Precision of positive class.

f1_p : float

F1-measure of positive class.

recall_n : float

Recall of negative class.

precision_n : float

Precision of negative class.

f1_n : float

F1-measure of negative class.

class eval_classifier.Validator(X_train, y_train, validator_type, obj_tsvm)[source]¶

Bases: object

It evaluates the TwinSVM model based on the specified evaluation method.

Parameters:

X_train : array-like, shape (n_samples, n_features)

Training feature vectors, where n_samples is the number of samples and n_features is the number of features.

y_train : array-like, shape (n_samples,)

Target values or class labels.

validator_type : tuple

A two-element tuple which contains type of evaluation method and its parameter. Example: (‘CV’, 5) -> 5-fold cross-validation, (‘t_t_split’, 30) -> 30% of samples for test set.

obj_tsvm : object

A TwinSVM model. It can be an instace of TSVM, MCTSVM, or OVO_TSVM.

Methods

`choose_validator`()	It selects the appropriate evaluation method based on the input paramters.
`cv_validator`(dict_param)	It evaluates the TwinSVM model using the cross-validation method.
`cv_validator_mc`(dict_param)	It evaluates the multi-class TwinSVM model using the cross-validation.
`split_tt_validator`(dict_param)	It evaluates the TwinSVM model using the train/test split method.

cv_validator(dict_param)[source]¶

It evaluates the TwinSVM model using the cross-validation method.

Parameters:

dict_param : dict

Values of hyper-parameters for the TwinSVM model.

Returns:

float

Mean accuracy of the model.

float

Standard deviation of accuracy.

dict

Evaluation metrics such as Recall, Percision and F1-measure for both classes as well as elements of the confusion matrix.

split_tt_validator(dict_param)[source]¶

It evaluates the TwinSVM model using the train/test split method.

Parameters:

dict_param : dict

Values of hyper-parameters for the TwinSVM model.

Returns:

float

Accuracy of the model.

float

Zero standard deviation.

dict

Evaluation metrics such as Recall, Percision and F1-measure for both classes as well as elements of the confusion matrix.

cv_validator_mc(dict_param)[source]¶

It evaluates the multi-class TwinSVM model using the cross-validation.

Parameters:

dict_param : dict

Values of hyper-parameters for the multiclss TwinSVM model.

Returns:

float

Accuracy of the model.

float

Zero standard deviation.

dict

Evaluation metrics such as Recall, Percision and F1-measure.

choose_validator()[source]¶

It selects the appropriate evaluation method based on the input paramters.

Returns:

object

An evaluation method for assesing the model’s performance.

eval_classifier.search_space(kernel_type, class_type, c_l_bound, c_u_bound, rbf_lbound, rbf_ubound, step=1)[source]¶

It generates all combination of search elements based on the given range of hyperparameters.

Parameters:

kernel_type : str, {‘linear’, ‘RBF’}

Type of the kernel function which is either ‘linear’ or ‘RBF’.

class_type : str, {‘binary’, ‘ovo’, ‘ova’}

Type of classification.

c_l_bound : int

Lower bound for C penalty parameter.

c_u_bound : int

Upper bound for C penalty parameter.

rbf_lbound : int

Lower bound for gamma parameter which is the hyperparameter of the RBF kernel function.

rbf_ubound : int

Upper bound for gamma parameter.

step : int, optinal (default=1)

Step size to increase power of 2.

Returns:

list

Search elements.

Examples

>>> from ltsvm import eval_classifier
>>> eval_classifier.search_space('RBF', 'binary', -1, 1, -1, 1)
[{'C1': 0.5, 'C2': 0.5, 'gamma': 0.5}...
{'C1': 1.0, 'C2': 1.0, 'gamma': 0.5}... {'C1': 2.0, 'C2': 2.0, 'gamma': 2.0}]

eval_classifier.grid_search(search_space, func_validator)[source]¶

It does grid search to find the optimcal values of hyperparameters for the TwinSVM model, which results in the best classfication accuracy.

Parameters:

search_space : list

All combination of search elements.

func_validator : object

An evaluation method for assesing the TwinSVM model’s performance.

Returns:

list

Classification results of the TwinSVM classifier using different set of hyperparameters.

eval_classifier.save_result(file_name, validator_obj, gs_result, output_path)[source]¶

It saves the detailed classification results in a spreadsheet file (Excel).

Parameters:

file_name : str

Name of the spreadsheet file.

validator_obj : object

The evaluation method that was used for the assesment of the TwinSVM classifier.

gs_result : list

Classification results of the TwinSVM classifier using different set of hyperparameters.

output_path : str

Path at which the spreadsheet file will be saved.

Returns:

str

Path to the saved spreadsheet (Excel) file.

eval_classifier.initializer(user_input_obj)[source]¶

It passes a user’s input to the functions and classes for solving a classification task. The steps that this function performs can be summarized as follows:

Specifies a TwinSVM classifier based on the user’s input.
Chooses an evaluation method for assessment of the classifier.
Computes all the combination of search elements.
Computes the evaluation metrics for all the search element using grid search.
Saves the detailed classification results in a spreadsheet file (Excel).

Parameters:

user_input_obj : object

An instance of UserInput class which holds the user input.