teacher.metrics#
The teacher.metrics module provides methods to measure
the performance of the different agnostic explainers from the
teacher.explanation module. These metrics take into account
factual explanations composed of multiple rules.
Functions#
The following functions are provided to compute different metrics:
coverage()Returns the coverage that a single rule has over a dataset. In fuzzy context, this means that an instance is covered by a rule if the minimum activation degree of any antecedent of the rule is higher than a given threshold. Then, the coverage will get all the instances that are covered at least by one rule of the ones composing the factual.
precision()Returns the percentage of the covered instances that match a given class value.
fidelity()This metric specific for agnostic classifiers measures the degree of similarity of the blackbox classifier and the whitebox classifier, by using the blackbox as ground truth and getting the score of the whitebox against that ground truth.
rule_fidelity()This metric checks the fidelity of the covered part of a dataset given a set of rules.
- teacher.metrics.coverage(rules, fuzzy_dataset, threshold=0.001)[source]#
Compute the fuzzy coverage of a dataset given a set of rules so that if a rule matches an instance above a threshold it is considered covered.
- Parameters:
fuzzy_dataset (dict) – Dataset membership with the format {feature_1: {set_1: [memb_1, memb_2, …]}} with each feature and set and an array of shape (n_instances) with t he membership degree of all the instances of the dataset
threshold (float, optional) – Activation threshold which sets when an instance is considered covered, by default 0.001
- Returns:
Ratio of number of covered instances divided by dataset length
- Return type:
- teacher.metrics.fidelity(y, y_local)[source]#
Compute the fidelity of an agnostic white-box classifier
- Parameters:
y (array-like, of shape (n_neighbor_instances)) – Black-box predictions for all the instances of the neighborhood
y_local (array-like, of shape (n_neighbor_instances)) – White-box predictions for all the instances of the neighborhood
- Returns:
Ratio of number of matching predictions
- Return type:
- teacher.metrics.implausibility(cf_instance, dataset, continuous, discrete, mad, distance='moth')[source]#
Return the level of plausibility of a counterfactual instance in the dataset
- Parameters:
cf_instance (array-like, 1D) – Reference instance
dataset (array-like, 2D) – Dataset with the instances to measure
continuous (array-like) – Indices of the continuous features
discrete (array-like) – Indices of the discrete features
mad (dict) – Median Absolute Distances of all the continuous features in the dataset, where the keys are the indices of the continuous features
- teacher.metrics.instability(instance, cf_instance, closest_instance, cf_closest_instance, continuous, discrete, mad, distance='moth')[source]#
Return the level of stability of a counterfactual instance against the counterfactual of the closest instance to the original instance
- Parameters:
instance (array-like, 1D) – Original instance
cf_instance (array-like, 1D) – Counterfactual applied original instance
closest_instance (array-like, 1D) – Closest instance to the original one
cf_closest_instance (array-like, 1D) – Counterfactual applied to the closest instance
continuous (array-like) – Indices of the continuous features
discrete (array-like) – Indices of the discrete features
mad (dict) – Median Absolute Distances of all the continuous features in the dataset, where the keys are the indices of the continuous features
- teacher.metrics.precision(rules, fuzzy_dataset, y, threshold=0.001)[source]#
Compute the precision of a dataset covered by set of rules to check if the class value matches the rule consequent.
- Parameters:
fuzzy_dataset (dict) – Dataset membership with the format {feature_1: {set_1: [memb_1, memb_2, …]}} with each feature and set and an array of shape (n_instances) with t he membership degree of all the instances of the dataset
y (array-like, of shape (n_instances)) – Class values corresponding to each instance of the dataset
threshold (float, optional) – Activation threshold which sets when an instance is considered covered, by default 0.001
- Returns:
Ratio of number of instances with same class value over the covered instances
- Return type:
- teacher.metrics.proximity_dissimilarity(instance, cf_instance, continuous, discrete, mad, distance='moth')[source]#
Compute the proximity dissimilarity between an instance and the applied counterfactual instance
- Parameters:
instance (array-like) – Original instance
cf_instance (array-like) – Counterfactual applied instance
continuous (array-like) – Indices of the continuous features
discrete (array-like) – Indices of the discrete features
mad (dict) – Median Absolute Distances of all the continuous features in the dataset, where the keys are the indices of the continuous features
- teacher.metrics.rule_fidelity(y, y_local, fuzzy_dataset, rules, threshold=0.001)[source]#
Compute the fidelity of an agnostic white-box classifier given a set of rules
- Parameters:
y (array-like, of shape (n_neighbor_instances)) – Black-box predictions for all the instances of the neighborhood
y_local (array-like, of shape (n_neighbor_instances)) – White-box predictions for all the instances of the neighborhood
fuzzy_dataset (dict) – Dataset membership with the format {feature_1: {set_1: [memb_1, memb_2, …]}} with each feature and set and an array of shape (n_neighbor_instances) with t he membership degree of all the instances of the dataset
threshold (float, optional) – Activation threshold which sets when an instance is considered covered, by default 0.001
- Returns:
Ratio of number of matching predictions from the covered instances divided by the number of covered instances
- Return type: