Learn about popular Explainable AI algorithms such as integrated gradient, LIME, class activation maps, counterfactual explanations, etc.

docker.tar.gz

Explainable AI is a set of tools and frameworks that helps you understand and interpret the internal logic behind the predictions made by a deep learning network. With this, you can generate insights into the behavior and working of the model to mitigate issues around it in the development phase.

In this course, you will be introduced to popular Explainable AI algorithms such as smooth gradient, integrated gradient, LIME, class activation maps, counterfactual explanations, feature attributions, etc., for image classification networks such as MobileNet-V2 trained on large-scale datasets like ImageNet-1K.

By the end of this course, you will understand the need for Explainable AI and be able to design and implement popular explanation algorithms like saliency maps, class activation maps, counterfactual explanations, etc. You will be able to evaluate and quantify the quality of the neural network explanations via several interpretability metrics.

Master Explainable AI: Interpreting Image Classifier Decisions

Which explanation is computed by these lines of code?
```s
    saliency = torch.zeros((1,1, MASK_SIZE, MASK_SIZE))
    mask_sum = torch.zeros((1,1, MASK_SIZE, MASK_SIZE))
    for i in range(NUM_MASKS):
        logits = model(mask*image.unsqueeze(0))
        prob = logits.detach().softmax(-1)[0, prediction]
        saliency[0,0] += mask[0, 0]*prob
        mask_sum[0,0] += mask[0,0]

    saliency = saliency/mask_sum
```

Which explanation is computed by these lines of code?
<pre><code class="language-s"> saliency = torch.zeros((1,1, MASK_SIZE, MASK_SIZE))
 mask_sum = torch.zeros((1,1, MASK_SIZE, MASK_SIZE))
 for i in range(NUM_MASKS):
 logits = model(mask*image.unsqueeze(0))
 prob = logits.detach().softmax(-1)[0, prediction]
 saliency[0,0] += mask[0, 0]*prob
 mask_sum[0,0] += mask[0,0]

 saliency = saliency/mask_sum
</code></pre>

Crisitims are data points that correspond to the extreme values of the witness function.

Crisitims are data points that result in almost zero value of the witness function.

Criticisms are data points not well represented by the set of prototypes. 

Criticisms are data points that represent the actual data distribution well.

**(Select all that apply.)** Which statements are _true_ about criticisms?

(Select all that apply.) Which statements are true about criticisms?

The density functions of the data and the prototypes are close together.

The prototype distribution overestimates the data distribution.

The prototype distribution underestimates the data distribution.

What can we infer if the witness function is positive for a particular data point $c$?

What can we infer if the witness function is positive for a particular data point <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>c</mi></mrow><annotation encoding="application/x-tex">c</annotation></semantics></math>c?

```s
    term1 = torch.sum(K[:][:, candidate_indices], 0)/NUM_SAMPLES
    term2 = torch.sum(K[prototype_indices, :][:, candidate_indices], 0)
    term2 /= prototype_indices.shape[0]
    term = (term1 - term2).abs()
```

```s  
    term1 = 2*torch.sum(K[:][:, candidate_indices], 0)/NUM_SAMPLES
    term2 = K.diagonal()[candidate_indices]
    term3 = 2*torch.sum(K[selected, :][:, candidate_indices], 0)
    term3 /= (selected.shape[0] + 1)
    term = term1 - term2 -term3
```

```s
    regularization = torch.sum(torch.abs(cf - image))
    top2_predictions = torch.topk(logits, 2)
    loss = nn.functional.relu(top2_predictions[0] - top2_predictions[1])
    objective = torch.mean(LAMBDA*loss + regularization)
```

Which code snippet computes the $MMD^2$ measure?

Which code snippet computes the <math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>M</mi><mi>M</mi><msup><mi>D</mi><mn>2</mn></msup></mrow><annotation encoding="application/x-tex">MMD^2</annotation></semantics></math>MMD2 measure?

```s
    grad = torch.autograd.grad(logits[0, prediction.item()], [f])[0]
    grad = grad.flatten().numpy()
```


```s 
   tcav = {k: np.mean(tcav[k]) for k in concepts}
```

```s 
    positives = concept_features[concept]
    negatives = torch.cat([concept_features[c] for c in concept_features if c!=concept])
    labels = torch.cat([torch.ones(positives.shape[0]), torch.zeros(negatives.shape[0])])
    
    data = torch.cat([positives, negatives])
    regr = LogisticRegression(random_state=0, fit_intercept=False).fit(data.numpy(), labels.numpy())
```

Which code snippet computes the CAV for a concept?

Which code snippet computes the CAV for a concept?

Quiz yourself on the concepts learned in this chapter.

Introduction to Explainable AI

Saliency Maps

Class Activation Maps

Miscellaneous Methods

Metrics of Interpretability

Final Assessment

Weakly Supervised Object Localization

Quiz on Miscellaneous Methods