Introduction to Explainable AI

Explainable AI, or XAI, is a tool that helps people understand and trust what a computer program powered by artificial intelligence is doing. Imagine you have a machine that can predict things, like whether you’ll get a loan or not. XAI helps make those predictions easier to understand. When we can’t understand or make sense of something, we usually don’t trust it. XAI is here to make sure we can trust the predictions made by AI. We use AI to make decisions that are fair and not influenced by personal biases. But these decisions can be risky if we can’t explain or understand them.

Why is Explainable AI important?

Explainable AI (XAI) plays a pivotal role in bridging the gap between artificial intelligence and business outcomes. It facilitates seamless communication between data science teams and nontechnical executives, fostering a shared understanding of product capabilities and limitations and ultimately enhancing governance.

Below, we look at the significance of Explainable AI:

Let’s look at some practical examples to explain why it matters:

• Medical diagnoses: Imagine a doctor using an AI system to diagnose diseases from medical scans. If the AI recommends a certain treatment, it’s crucial for the doctor to understand why that recommendation was made. Explainable AI can provide clear reasons, helping the doctor make better treatment decisions.

• Loan approvals: Banks use AI to decide whether to approve loans for customers. If a person’s loan is denied, they’d want to know why. Explainable AI can show which factors influenced the decision, helping the customer understand and possibly improve their chances in the future.

• Online shopping recommendations: E-commerce platforms employ recommendation algorithms to suggest products to customers. By providing explanations for these recommendations, businesses can enhance customer trust and satisfaction.

• Regulatory compliance: Many industries, such as pharmaceuticals, are subject to strict regulatory requirements. In this scenario an AI-driven drug discovery system can provide understandable explanations for why certain compounds are selected for further testing.

In all these cases, Explainable AI ensures that the decisions made by AI systems are understandable, fair, and accountable. It helps build trust between humans and AI, leading to better outcomes and reduced risk of biased or unjust decisions.

Explainable AI trade-offs

Explainable AI (XAI) involves several trade-offs that need to be carefully considered when implementing it. Here are some key trade-offs in Explainable AI:

• Accuracy vs. interpretability: Highly accurate models can be complex and challenging to interpret, while simpler, more interpretable models may sacrifice accuracy. For example, in healthcare, a complex deep learning model may achieve high accuracy in diagnosing diseases from medical images, but it’s difficult to explain why it made a specific diagnosis. In contrast, a simpler decision tree model might be less accurate but provides clear, interpretable rules for diagnosis.

• Complexity vs. simplicity of explanations: More complex models might require sophisticated explanations, increasing the complexity of the explanation process. For example, in financial fraud detection, an intricate ML model might detect subtle patterns in transaction data but require complex explanations involving multiple variables. A simpler model, like a rule-based system, can provide straightforward explanations but may miss some nuanced fraud patterns.

• Granularity of explanations: Providing detailed local explanations for individual predictions might not generalize well to global model behavior. For example, in an e-commerce recommendation system, a local explanation for why a particular product was recommended to a user might involve their recent browsing history. However, this local explanation might not capture the broader patterns of recommendation behavior across all users.

What does explainability mean for different personas?

The explainability of a model can mean different things to different people depending on how they are using the AI solution. Therefore, it is important that we try to understand the user persona who is seeking the model explanations.

The initial step in selecting the appropriate tool lies in comprehending the various forms of explanations available. Rather than resorting to a trial-and-error approach by experimenting with multiple techniques until an explanation appears satisfactory, which is akin to seeking investment advice from random strangers on the street, it is far more effective to identify the explanation type most suited to the business needs. To determine this, start by considering the recipients of the explanation, the subject matter that necessitates clarification, and the anticipated outcome post-explanation.

For instance, when delivering an explanation to an end user who has been denied a loan, a global explanation delving into the model’s architecture may not prove relevant or actionable. It would be wiser to employ a localized technique that focuses on specific features, aligning the explanation with factors within the user’s control. In contrast, business stakeholders may have distinct requirements and preferences when it comes to explanations.

Let’s study a few personas below and see the requirement from explainability for each:

Supporting frameworks

Several popular tools and libraries are available for implementing XAI techniques and enhancing the interpretability of machine learning models. Here are some of the well-known ones: