Learn about popular Explainable AI algorithms such as integrated gradient, LIME, class activation maps, counterfactual explanations, etc.

docker.tar.gz

Explainable AI is a set of tools and frameworks that helps you understand and interpret the internal logic behind the predictions made by a deep learning network. With this, you can generate insights into the behavior and working of the model to mitigate issues around it in the development phase.

In this course, you will be introduced to popular Explainable AI algorithms such as smooth gradient, integrated gradient, LIME, class activation maps, counterfactual explanations, feature attributions, etc., for image classification networks such as MobileNet-V2 trained on large-scale datasets like ImageNet-1K.

By the end of this course, you will understand the need for Explainable AI and be able to design and implement popular explanation algorithms like saliency maps, class activation maps, counterfactual explanations, etc. You will be able to evaluate and quantify the quality of the neural network explanations via several interpretability metrics.

Master Explainable AI: Interpreting Image Classifier Decisions

Given a post-hoc explanation $S = [0.1, 0.4, -0.6, 0.7, 0.9, -0.5]$ and the ground-truth explanation $S_g = [0.2, 0.3, -0.4, 0.9, 0.4, -0.5]$, what is the Top-5 feature agreement metric between them?

Given a post-hoc explanation <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>S</mi><mo>=</mo><mo stretchy="false">[</mo><mn>0.1</mn><mo separator="true">,</mo><mn>0.4</mn><mo separator="true">,</mo><mo>−</mo><mn>0.6</mn><mo separator="true">,</mo><mn>0.7</mn><mo separator="true">,</mo><mn>0.9</mn><mo separator="true">,</mo><mo>−</mo><mn>0.5</mn><mo stretchy="false">]</mo></mrow><annotation encoding="application/x-tex">S = [0.1, 0.4, -0.6, 0.7, 0.9, -0.5]</annotation></semantics></math>S=[0.1,0.4,−0.6,0.7,0.9,−0.5] and the ground-truth explanation <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>S</mi><mi>g</mi></msub><mo>=</mo><mo stretchy="false">[</mo><mn>0.2</mn><mo separator="true">,</mo><mn>0.3</mn><mo separator="true">,</mo><mo>−</mo><mn>0.4</mn><mo separator="true">,</mo><mn>0.9</mn><mo separator="true">,</mo><mn>0.4</mn><mo separator="true">,</mo><mo>−</mo><mn>0.5</mn><mo stretchy="false">]</mo></mrow><annotation encoding="application/x-tex">S_g = [0.2, 0.3, -0.4, 0.9, 0.4, -0.5]</annotation></semantics></math>Sg​=[0.2,0.3,−0.4,0.9,0.4,−0.5], what is the Top-5 feature agreement metric between them?

For an input $X=[0.1, -0.1]$, the explanation returned by an algorithm is $S=[0.5, -0.5]$. Compute the relative input stability of the algorithm when the following is given:

- When $X$ is perturbed to $X_1= [0.2, 0.0 ]$, the explanation changes to $S_1 = [0.6, -0.8]$.
- When $X$ is perturbed to $X_2 = [0, -0.2 ]$, the explanation changes to $S_2 = [0.4, -0.4]$.

For an input <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>X</mi><mo>=</mo><mo stretchy="false">[</mo><mn>0.1</mn><mo separator="true">,</mo><mo>−</mo><mn>0.1</mn><mo stretchy="false">]</mo></mrow><annotation encoding="application/x-tex">X=[0.1, -0.1]</annotation></semantics></math>X=[0.1,−0.1], the explanation returned by an algorithm is <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>S</mi><mo>=</mo><mo stretchy="false">[</mo><mn>0.5</mn><mo separator="true">,</mo><mo>−</mo><mn>0.5</mn><mo stretchy="false">]</mo></mrow><annotation encoding="application/x-tex">S=[0.5, -0.5]</annotation></semantics></math>S=[0.5,−0.5]. Compute the relative input stability of the algorithm when the following is given:
<ul>
<li>When <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>X</mi></mrow><annotation encoding="application/x-tex">X</annotation></semantics></math>X is perturbed to <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>X</mi><mn>1</mn></msub><mo>=</mo><mo stretchy="false">[</mo><mn>0.2</mn><mo separator="true">,</mo><mn>0.0</mn><mo stretchy="false">]</mo></mrow><annotation encoding="application/x-tex">X_1= [0.2, 0.0 ]</annotation></semantics></math>X1​=[0.2,0.0], the explanation changes to <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>S</mi><mn>1</mn></msub><mo>=</mo><mo stretchy="false">[</mo><mn>0.6</mn><mo separator="true">,</mo><mo>−</mo><mn>0.8</mn><mo stretchy="false">]</mo></mrow><annotation encoding="application/x-tex">S_1 = [0.6, -0.8]</annotation></semantics></math>S1​=[0.6,−0.8].</li>
<li>When <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>X</mi></mrow><annotation encoding="application/x-tex">X</annotation></semantics></math>X is perturbed to <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>X</mi><mn>2</mn></msub><mo>=</mo><mo stretchy="false">[</mo><mn>0</mn><mo separator="true">,</mo><mo>−</mo><mn>0.2</mn><mo stretchy="false">]</mo></mrow><annotation encoding="application/x-tex">X_2 = [0, -0.2 ]</annotation></semantics></math>X2​=[0,−0.2], the explanation changes to <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>S</mi><mn>2</mn></msub><mo>=</mo><mo stretchy="false">[</mo><mn>0.4</mn><mo separator="true">,</mo><mo>−</mo><mn>0.4</mn><mo stretchy="false">]</mo></mrow><annotation encoding="application/x-tex">S_2 = [0.4, -0.4]</annotation></semantics></math>S2​=[0.4,−0.4].</li>
</ul>

The fraction of Top-k features that are common between a given post-hoc
explanation and the corresponding ground-truth explanation.

The fraction of Top-k features that are not only common between a given post-hoc explanation and the corresponding ground-truth explanation but also have the same position in the respective rank orders.

The maximum change in explanation relative to changes in the inputs.

The difference in prediction probability that results from perturbing the features deemed as influential by a given post-hoc explanation.

What does the Prediction Gap on Important feature perturbation (PGI) measure?

What does the Prediction Gap on Important feature perturbation (PGI) measure?

 ```s
all_feat_ranksA = rankdata(-np.abs(attrA), method='dense')
all_feat_ranksB = rankdata(-np.abs(attrB), method='dense') 

metric_distr, _ = pearsonr(all_feat_ranksA, all_feat_ranksB)
```

```s
topk_idA = np.argsort(-np.abs(attrA))[0:k]
topk_idB = np.argsort(-np.abs(attrB))[0:k]   

topk_idA = np.argsort(-np.abs(attrA))[0:k]
topk_idB = np.argsort(-np.abs(attrB))[0:k]

metric_distr = len(topk_setsA.intersection(topk_setsB))/k
```

  ```s
topk_setsA = set([(idx, sign) for (idx, sign) in zip(topk_idA, topk_idA_sign)])
topk_setsB = set([(idx, sign) for (idx, sign) in zip(topk_idB, topk_idB_sign)])
    
metric_distr = len(topk_setsA.intersection(topk_setsB))/k
```

Which code snippet computes the feature agreement (FA) metric?

Which code snippet computes the feature agreement (FA) metric?

```s
explanations_diff = torch.stack(explanations_diff).norm(2, -1)
inputs_diff =  torch.stack(inputs_diff).norm(2, -1)

metric = explanations_diff/inputs_diff
return metric.max()/1e6
```

```s
topk_setsA = set([(idx, sign) for (idx, sign) in zip(topk_idA, topk_idA_sign)])
topk_setsB = set([(idx, sign) for (idx, sign) in zip(topk_idB, topk_idB_sign)])
    
metric_distr = len(topk_setsA.intersection(topk_setsB))/k
return metric_distr
```

```s  
metric = torch.abs(logits_peturbed[:, prediction] - logits[:, prediction])
return metric.mean().item()
```

Which code snippet computes the PGI metric?

Which code snippet computes the PGI metric?

Quiz yourself on the concepts learned in this chapter.

Introduction to Explainable AI

Saliency Maps

Class Activation Maps

Miscellaneous Methods

Metrics of Interpretability

Final Assessment

Weakly Supervised Object Localization

Quiz on Metrics of Interpretability